Digital Typhoon: About Database of Weather Charts for Hundred Years
1. About Database of Weather Charts for Hundred Years
Weather charts are valuable historical resources for the history of meteorological observations in Japan, because they have been the key material to understand and predict weather based on the integration and visualization of meteorological observations of the ground and upper atmosphere. We therefore started the construction of a long-term data archive for weather charts that have been created by Japan Meteorological Agency (JMA) and its predecessors since March 1, 1883 for more than 130 years, and released a website called “Database of Weather Charts for Hundred Years.”
Data acquisition and release
The core part of this database is weather charts purchased from Japan Meteorological Business Support Center. Weather charts were then uniformly extracted and converted, so that they can be searched by date. Continuous from the beginning is the surface weather chart around Japan or Asia Pacific, and the type of weather charts has been gradually increasing to reach eight at the latest. From the viewpoint of time series, a small number of weather charts are randomly missing, but as a whole it can be considered as continuous time series. But weather charts are significantly missing for twenty days in September 1923, which seems to be the result of fire at the Central Meteorological Observatory in Tokyo after Great Kanto Earthquake on September 1.
As of February 2016, we have 108,599 surface weather charts, and the total of 232,298 weather charts for all types, and the website was publicly released on November 2015, after experimental release on January 2014. The website shows a credit stating that this database is based on the usage of data from JMA according to the data policy of JMA.
Organization of the database
The original plan of this database is to construct the database of weather charts that are geometrically corrected (georeferenced). The manual geometric correction of weather charts is infeasible for more than 100,000 weather charts, so the key is the development of an automated process for the geometric correction of surface weather charts. Weather charts after August 1958 can be processed to a satisfactory level, so we take advantage of geobrowsers such as Google Earth, Google Maps and Cesium to show the overlay of georeferenced weather charts. On the other hand, weather charts before July 1958 have a problem of low quality due to preservation and a larger variation of format, so we still do not have a fixed plan for georeferencing.
The next challenge is to improve the usability of the database. Search by date is the first method to implement in the database, but the problem here is the lack of advanced search due to the lack of metadata. We implemented four methods to improve the findability of weather charts.
We added search for the description of “daily weather charts” from JMA to enable keyword search on relevant weather phenomena.
We integrated with other databases of “Digital Typhoon,” to provide a variety of access methods such as weather charts of the day of typhoon landfalls, or weather charts of the day of disasters that satisfy a set of criteria.
We created short narratives on relevant weather charts with the help of NPO Weather Caster Network to access weather charts of historical events and extreme observations (29 cases) or major typhoons (19 cases) through the narrative of weather charts.
We used an image browser with synchronized timelines called “SyncReel,” to offer efficient access to past weather charts by scrolling horizontal timelines.
The challenge for the future is the mining of information embedded within the weather chart. Automatic extraction of text by OCR is possible for text documents, but the same method does not work for weather charts on which lines are written multiple times. Hence we should take advantage of the methodology of crowd-sourcing or citizen science, where scientists and citizens can take part in the activity of information extraction from weather charts. Several successful projects have been reported in this direction, but the application to this project should solve the following issues. First, we need training for the proper interpretation of data such as different unit of pressure. Second, we need to clarify contribution to science about how historical weather charts can contribute to scientific discovery, because this is where citizens are motivated to contribute to the activity.
Other meteorological data archives
The archive of meterological observation data, the foundation of weather charts, is available on Digital Typhoon as AMeDAS Statistics - Analysis of Past AMeDAS Data and Information on AMeDAS Stations after 1976 when the ground observation has changed to AMeDAS system. On the other hand, the archive of meteorological satellite data was the original purpose of Digital Typhoon, and our database provides satellite data since December 1978, which is the longest archive in Japan as the archive of Himawari satellite series.
Weather charts in the database are originally provided from Japan Meteorological Agency (JMA) through Japan Meteorological Business Support Center (JMBSC). The usage of data should follow the data policy of Japan Meteorological Agency. An example of credit is as follows.
Created by National Institute of Informatics "Digital Typhoon" based on "Weather Charts" from Japan Meteorlogical Agency
This site provides two types of images, (1) images transformed in size and format from the original images, (2) images converted by approximate geometrical transoformation (including errors) to match with maps. The latter can be viewed on Google Maps, Google Earth and Cesium.
This database does not provide the comprehensive collection of images contained in the original DVD, and the processing method is not verified. Users are requested to refer to the original DVD from JMA for strict research.
The database was supported by Grant-in-Aid for Publication of Scientific Research Results; Ministry of Education, Culture, Sports, Science and Technology (MEXT) for 2013 (No. 258062). Please see About Database of Weather Charts for Hundred Years about the detail of the database.
New weather charts until October 2016 were added to increase the total to 238,154.
New weather charts until August 2016 were added to increase the total to 237,178.
- New weather charts were added to become the total number of weather charts to 234,234.
Database of Weather Charts for Hundred Years: Construction of long-term data archives on weather charts created by Japan Meteorological Agency was presented.
The archive of the "Spirit of Observation" - 'Database of weather charts for hundred years' and the history of meteorological observations was released for the 133th year anniversary on the publication of weather charts.
About Database of Weather Charts for Hundred Years was released.
- New weather charts were added, and some weather charts missing due to a trouble in the process were also added, resulted in the increase of weather charts from 229,431 to 232,298.
Weather Charts Calendar Search was released.
- Search methods was enhanced including links to historical weather.
- The coverage of weather charts is extended to the total of 228,935 charts from March 1883 to June 2015, and georeferenced weather charts are visualized on Google Maps.
- Release of Asia Pacific surface weather chart from October 2014 to June 2015.
- Release of Notable Weather Charts in the Past (in Japanese).
- Release of Daily Weather Charts Comment Search (1996-2013) (in Japanese).
- Release of Asia Pacific surface weather chart from October 2013 to September 2014.
- Weather charts can be browsed within a browser using Cesium. This is a response to the termination of Google Earth Plugin.
- Release of Asia Pacific surface weather chart from January 1997 to December 1999; total 23,880 charts.
- Release of Asia Pacific surface weather chart from August 2013 to September 2013; total 19,502 charts.
SYNCREEL was released.
- Release of Asia Pacific surface weather chart from January 2000 to February 2006; total 19,256 charts.
- Release of Asia Pacific surface weather chart from March 2006 to July 2013; total 10,548 charts.