# Library needed to deal with DataFrames
import pandas as pdInteractive, Informative Maps of the Automated Meteorological Data Acquisition System (AMeDAS) Weather Stations in Japan
(A) Data Collection
(B) Data Preprocessing
(D) Data Visualization
weather data
Japan Meteorological Agency
data scraping
data parsing
library - requests
library - bs4
library - pandas
Abstract
Searching for the URL of a specific weather station when data scraping from the Japan Meteorological Agency (JMA) website can be time consuming. Especially when the target coordinates are close to a prefectural border, the interface provided on the JMA website is often inadequate. In this project, I present an interactive, informative map that helps data scrapers to quickly locate the target weather station and the parameters needed to designate its URL. The map displays information of the Automated Meteorological Data Acquisition System (AMeDAS) weather stations which was downloaded as a CSV file from the JMA website, combined with the URL information scraped from links on the JMA’s map interface. The data was preprocessed by romanizing the Japanese station names and converting the Japanese calendar to a Gregorian calendar. The data were plotted as an interactive map using Python’s folium library. The resulting project dashboard provides two maps. The smaller map of Japan shows all 1316 weather stations colour-coded to its appropriate jurisdiction. The larger map displays one jurisdictional map in each tab. By clicking a weather station on the jurisdictional map, an informative pop-ups is displayed. The pop-up provides additional information including a small table summarizing the type of weather data recorded at the station, the date that data collection began, the coordinates and elevation of the site, and the height of the thermometer and anemometer. The map helps data scrapers by providing a quick and easy way of accessing the information needed to designate the JMA station URL.
Warning
This project is still ongoing. Documentation is incomplete.
Snapshot of the interactive and informative maps of all AMeDAS weather stations from the Japan Meteorological Agency. Click here to see the maps in action!
1 So, What Am I Up To?
- Translating my lab notes and code from my exploratory analysis into clean, concise documentation
- Scraping data from URL links to identify a JMA stations’s
prec_noandblock_no - Reading a file of unknown encoding
- Converting Japanese ‘reki’ dates into Gregorian equivalents
- Converting names of places from Japanese to romaji (using English alphabet)
- Drawing stations on an interactive map using
folium- Generating simple HTML for legends, mouse-over text, and pop-up text
2 Introduction
Japan has an extensive weather data collection network, built by the Japan Meteorological Agency (JMA). The foundation for their network is the automatic collection and recording of various weather data at fixed time intervals, called the Automated Meteorologcial Data Acquisition System (AMeDAS). The system started its operation in 1974, and the most basic AMeDAS stations record just rainfall, but are placed at an average interval of 17 km all over Japan1. Fifty years later, as of July 2026, there are a total of 1316 AMeDAS stations in Japan, which record rainfall, wind direction and speed, temperature, relative humidity and even snowfall at some sites1.
While the comprehensive data collection network provides an abundance of weather data with high geographical precision, looking for the weather station closest to a desired location can become a time-consuming task. Especially when the target coordinates are close to a prefectural border, the interface provided on the JMA website is often inadequate. On the other hand, JMA’s AMeDAS observation map displays all weather stations as one map, as seen in Figure 1. The map and provides quick access to recent weather data, but it is not much useful in data scraping as it does not provide the URL to past meteorological records.
In this project, I present an interactive, informative map that helps data scrapers to quickly locate the target weather station and the parameters needed to designate its URL. The maps inform the user of the weather station identifiers that constitute parts of the URL, the type of meteorological data that are available from the site, and when data collection of these data began. All data used in this project were available from the JMA website, either as downloadable CSV files or directly scraped from their web source code. As the CSV file as designed with a Japanese reader in mind, I preprocessed the table to make an English-friendly version and saved the resulting pandas DataFrame. Namely, Japanese station names were converted into romaji (ie. using English alphabets), and the Japanese calendar dates were changed to a Gregorian calendar. The folium library was used to draw the interactive maps, which provide a quick and easy way of accessing the information needed to designate the JMA station URL.
3 Requirements
I have provided all code involved in this project in my GitHub repository. The most informative file is the map-japan-weather-stations-deets-n-docs.ipynb Jupyter notebook, which explains the steps taken from start to finish. I have also provided a CSV file containing the preprocessed data in the data/ subdirectory. Readers not interested in data collection and preprocessing may go straight to the map generation with the map-japan-weather-stations-dashboard.ipynb Jupyter notebook.
Here is a list of things you’ll need to follow along with how I made the maps.
3.1 Requirements for Running Project From Start to Finish
- A copy of either the
map-japan-weather-stations-deets-n-docs.ipynbJupyter notebook ormap-japan-weather-stations-data-collection-preprocessing.pyPython script from my GitHub repository - A copy of
map-japan-weather-stations-dashboard.ipynbJupyter notebook from my GitHub repository data/subfolder for saving and loading CSV files- Python libraries
requestsbs4pandasfolium- If
foliummaps do not render in your IDE, you’ll want the following libraries as well:base64IPython
3.2 Requirements for Just Data Visualization with Maps
If you’re here just to find out how I made the folium maps, you can skip the data collection and preprocessing steps. Here are the things you’ll need.
- A copy of
map-japan-weather-stations-dashboard.ipynbJupyter notebook from my GitHub repository data/subfolder containing the following files from my GitHub repositoryamedas-stations-all.csv
- Python libraries
pandasfolium- If
foliummaps do not render in your IDE, you’ll want the following libraries as well:base64IPython
4 Methods
The general steps involved in this project are as follows:
- Collect data
- Download tabular data on AMeDAS weather stations as a CSV file from the JMA website and [read it as a
pandasDataFrame by using the proper encoding] - Scraping and parsing
prec_noandblock_nofrom the JMA website’s map interface
- Download tabular data on AMeDAS weather stations as a CSV file from the JMA website and [read it as a
- Preprocess data
- Convert latitude and longitude from degrees and minutes to decimals
- Make a romanized (alphabet) version of the JMA station names
- Convert the dates that recordings started from Japanese calendar to the Gregorian calendar
- Note which data were recorded at each station
- Save the preprocessed data as a CSV file
- Visualize the weather stations on a map
foliummap of all 1316 JMA stations with mouseover information- full map could not handle all 1316 stations with mouseover and popup
- circleMarker was used to further simplify the map
foliummap of each jurisdiction with mouseover and popup information- legend written with some HTML
I’ll start with the data collection step.
4.1 Data Collection
I collected two pieces of information, both from the JMA website.
The first data was a CSV file containing information on all current AMeDAS stations in a table format. The file was easily extracted from a downloadable ZIP file on the JMA website, but required some minor deciphering to open the file with the appropriate encoding.
The second data was information on the URL bits needed to access each weather station. This was scraped from the source code on the JMA map interface, and saved as a dictionary for later access.
4.1.1 Downloading the Table Summary of All AMeDAS Stations
Information about current and past AMeDAS stations, including its ID number, its type, its location and when it started recording, are available in PDF or ZIP file (containing a CSV file) on the Japan Meteorological Agency (JMA) website. I have also provided a copy of the CSV file (ame_master_20260324.csv) under the data/ subdirectory in my GitHub repository. The last part of the file name, _20260324.csv, indicates that the JMA last updated the CSV file on March 24, 2026, most likely reflecting an addition or change in one or more of its AMeDAS weather stations. Depending on when you are reading this documentation, JMA may have a newer version of the file.
If you would like to check out the latest version of the file, head over to their website, scroll down until you see ‘観測所一覧’ (list of all stations). The latest date that the files were changed are indicated by the date beside ‘最新ファイル’ (newest file). If a newer file than the one I’ve provided is available, you can download the up-to-date file by clicking the second link as shown in red in Figure 2.
Note
For those wondering what the ZIP file in the third link contains, it deals with just the weather stations recording snowfall. As of July 2026, all weather stations recording snowfall also record other meteorological data, so they are contained within the second ZIP file.
Downloading and extracting the CSV file ame_master_20260324.csv is quite easy. However, opening the file can be a bit tricky. Specifically, I encountered an UnicodeDecodeError, and needed to pinpoint the encoding model used by the CSV file. For those interested in more information, I’ve summarized the steps I took to open a CSV file of unknown encoding in this short blog article.
For everyone else, I’ll just cut to the chase and tell you that the file encoding was 'CP932'. This was used to read the CSV file into a DataFrame using pd.read_csv() method from the pandas library.
# Read the CSV file as a pandas DataFrame
fileName = './data/ame_master_20260324.csv'
detected_encoding = 'CP932'
amedas_df = pd.read_csv(fileName, encoding=detected_encoding)The .head() method confirms the file has been successfully decoded.
amedas_df.head()| 都府県振興局 | 観測所番号 | 種類 | 観測所名 | ｶﾀｶﾅ名 | 気象情報等に表記する名称 | 所在地 | 緯度(度) | 緯度(分) | 経度(度) | 経度(分) | 海面上の高さ(ｍ) | 風速計の高さ(ｍ) | 温度計の高さ(ｍ) | 観測開始年月日 | 備考1 | 備考2 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 宗谷 | 11001 | 四 | 宗谷岬 | ｿｳﾔﾐｻｷ | 稚内市宗谷岬 | 稚内市宗谷岬 | 45 | 31.2 | 141 | 56.1 | 26 | 10.1 | 3.1 | 昭53.10.30 | － | － |
| 1 | 宗谷 | 11016 | 官 | 稚内 | ﾜｯｶﾅｲ | 稚内市開運 | 稚内市開運 稚内地方気象台 | 45 | 24.9 | 141 | 40.7 | 3 | 24.1 | － | #昭50.4.1 | 11903 | － |
| 2 | 宗谷 | 11046 | 四 | 礼文 | ﾚﾌﾞﾝ | 礼文町香深 | 礼文郡礼文町大字香深村トンナイ | 45 | 18.3 | 141 | 2.7 | 65 | 9.9 | 3.1 | 平15.10.17 | － | － |
| 3 | 宗谷 | 11061 | 官 | 声問 | ｺｴﾄｲ | 稚内空港 | 稚内市大字声問村字声問 稚内航空気象観測所 | 45 | 24.2 | 141 | 48.1 | 8 | 10 | － | 平15.1.1 | 11904 | 日照・湿度・気圧を除く |
| 4 | 宗谷 | 11076 | 四 | 浜鬼志別 | ﾊﾏｵﾆｼﾍﾞﾂ | 猿払村浜鬼志別 | 宗谷郡猿払村浜鬼志別 | 45 | 20.1 | 142 | 10.2 | 13 | 9.9 | 2.7 | 昭53.10.30 | 11900 | － |
This table contains the majority of the information that was drawn out on the map, including the latitude and longitude of the weather stations, the name of the weather stations, when data collection began at each site, the types of weather stations (which hint at the type of meteorological data collected at the site). However, some of the data is written in Japanese, the dates are noted in the Japanese calendar, and there are certain “exceptions” that need to be considered when scanning for the type of weather data recorded at the station.
In other words, the data needs to be cleaned.
However, even before cleaning the data in the DataFrame, I needed one more bit of information. Namely, the bits and pieces of the URL that are able to pinpoint specific JMA weather stations. The actual information that are useful when scraping data from multiple weather stations.
Important
Thou shall not pass!
…unless you’re okay with messy notes and half-finished documentation that you’ll encounter from this point onwards.
4.1.2 Data Scraping the prec_no and block_no from Links on the Map Interface
Data scraping prec_no and block_no from the JMA website’s map interface
5 Results
The smaller map of Japan shows all 1316 weather stations colour-coded to its appropriate jurisdiction. The larger map displays one jurisdictional map in each tab. By clicking a weather station on the jurisdictional map, an informative pop-ups is displayed. The pop-up provides additional information including a small table summarizing the type of weather data recorded at the station, the date that data collection began, the coordinates and elevation of the site, and the height of the thermometer and anemometer.
Snapshot of the interactive and informative maps of all AMeDAS weather stations from the Japan Meteorological Agency. Click here to see the maps in action!
6 Discussion
6.1 Limitations
- Romanized version of the station names do not always agree with the commonly known name.
- For example, “トウキョウ” is translated as “Toukyou” or “Tokyou”, while its commonly known name is “Tokyo”
- Similarly, “オオサカ” is displayed as “Oosaka” in my map, but is usually written as “Osaka”
- JMA weather stations that only collect a subset of meteorological data, but have the same station name have slightly inaccurate pop-up information
- The data collection equipment of some stations seem to be located at nearby, but different locations.
- For example, the ‘Obihiro’ station in under the Sapporo jurisdiction has one site collecting temperature, rainfall and snowfall, and a nearby site (with the same name but different coordinates) collecting wind and sunshine data
- For such sites, the tabular data of which data is collected at which site is somewhat mixed. In most cases, all data collected at the other site will also be displayed as a ‘Y’ on the table, with the exception for snowfall data.
- In other words, some JMA weather stations are mapped at two coordinates, with slightly inaccurate table data in the pop-ups.
6.2 Future Directions
7 Conclusions
8 References
1.
Japan Meteorological Agency. 地域気象観測システム（アメダス）. 気象庁 (2026).
9 Further Readings
- Getting a
UnicodeDecodeErrorwhen opening a mystery CSV file? Check out this blog post on reading a CSV file with unknown encoding into apandasDataFrame - Read this blog post to transcribe or transliterate Japanese words into the Roman alphabet
- Are Japanese dates giving you headaches? Refer to this blog article on how Japanese dates work and how to convert them to Gregorian dates