Beruflich Dokumente
Kultur Dokumente
IT’S
MADE
The Ultimate Guide to
Weather Forecasting
CONTENTS
Introduction............................................................................................................................................. 5
Weather Stations............................................................................................................................... 16
How weather experts improve the data: The six-step process to improve observational data.............. 18
Weather Radar................................................................................................................................... 20
Weather Satellites.............................................................................................................................. 24
Lightning Detection........................................................................................................................... 28
What is the difference between external and in-house metocean models? .......................................... 42
Model Output Statistics (MOS): What is it & How it Contributes to Your Forecast......................... 50
How the MOS adds value to weather data to create a more accurate forecast....................................... 53
8 industries = 8 approaches.................................................................................................................... 90
Conclusion.............................................................................................................................................. 100
Contributions......................................................................................................................................... 108
INTRODUCTION
When it comes to weather forecasting, many people rely on freely
available sources. But predicting the weather is serious business.
Everyone from shipping companies, oil rigs, energy distributors, air
transport, the automotive industry, and more: they all depend on
accurate and reliable weather forecasts to make informed business
decisions.
6
The Weather Forecasting
Methodology
7
Introduction
The Five
Categories to
Create a Highly
Accurate Weather
Forecast
The weather experts agree, there are Five Categories that are essential to
a high-quality, professional forecasting service. To start, we’ll outline the
Five Categories, before diving into more detail in the upcoming chapters.
#1. Observations
You can tell a lot about the weather outside by looking out your window.
It’s kind of what meteorologists do. They employ thousands of weather
stations and other sensors all around the world to find out what’s going
on. There are two main types of observation networks:
Physical locations: For example, weather stations (on land) and buoys
(on water) but also modern IoT techniques to capture data e.g. from car
sensors
9
#3. Statistical post processing
10
Securing quality over all elements of the forecasting process can be
a challenge when it comes to something as variable as the weather,
meaning there’s a strong need for quality control. For example, incoming
observation data has to be checked on accuracy, completeness, and
irregularities. Forecasts have to be checked in a continuous way so
that models can be further improved and the level of accuracy towards
customers can be increased secured and increased.
To truly “know” the weather, you first need both weather forecasters
and data specialists. Second, you need a services team that deals with
customer feedback and draw up reports. Third, you need a research team
that innovates new scientific and technical methods, develops customer-
specific solutions, and defines algorithms to verify forecasting quality.
This research team is also responsible for knowledge management, for
example by monitoring weather model performance and keeping track of
new developments and improvements.
You might say accuracy inevitably comes with high maintenance, which
is why companies choose to work with specialist weather experts. But
utilizing this accuracy is what sets apart the good from the great. Average
companies can make decisions using average data, but leading companies
use the highest accurate weather data in their decision making processes.
11
Category One:
WEATHER
OBSERVATION
NETWORKS
Monitoring the weather is part of many people’s daily routine at work.
They do this to understand the safety risks for colleagues, how the
weather can impact operations, and understand the steps required to
keep the business running profitably.
There are two main types of observation networks used to help create a
weather forecast:
1. Physical locations - e.g., weather stations (on land) and buoys (on
water), which measure conditions in their precise location
13
#1. Weather stations
Weather stations provide observation data readings of atmospheric
conditions at their physical location. The data provided will depend on
where the station is based. Marine buoys, for instance, will give nautical
weather information like wave height. Road sites provide accurate
measurements of road conditions.
14
#4. Lightning detection
During a thunderstorm, every lightning strike creates electromagnetic
waves that travels through the atmosphere at the speed of light. Ground-
based (terrestrial) antenna networks can detect these waves. Regional
networks play a vital role in accurately identifying lightning with
terrestrial systems and satellites.
15
Category One:
Weather Observation Networks
Weather
Stations
Weather stations are physical locations equipped with standardized
instruments to measure the local weather conditions. Primarily, they
measure precipitation, wind speed, temperature, cloudiness, visibility
and other atmospheric conditions.
Two types of data are available from weather stations: observed data and
converted data. Observed data are collected from a range of different
weather stations, including:
• Marine buoys and platforms provide nautical weather data, e.g. wave
height
17
Converted data are transformed from observation data and include:
Weather experts will only use reliable data sources, like local Met Offices.
Where possible, they invest in multiple sources to correlate results. This
approach helps to improve the location coverage and quality of data.
They will also complete the observational data from weather stations
with other sources to create an accurate view.
Step 1: Preparing the data for processing, where the data is decoded and
checked for quality
Step 3: Unique derived elements are added, for example, cloud cover
18
How to identify accurate weather
stations sources
Weather station data can be categorized into three quality bands: High,
moderate, and uncertain. These bands help identify which sources are
more likely to be accurate and reliable.
They’ll use this data to identify situations where the weather can pose a
risk to safety, impact on the day-to-day operations, and understand the
steps required to keep the business running profitably
19
Category One:
Weather Observation Networks
Weather Radar
20
Weather radar uses radio waves to locate precipitation. It calculates
the distance, the direction, and elevation of precipitation. Additionally,
some radars provide the type of precipitation (e.g. rain, snow or hail).
Currently, radar data is primarily available from national met offices,
which operate radar networks.
During the Second World War, radar operators noticed a background echo
showing up on their displays. Further investigation showed it was the
weather causing this interference.
“Good radar data is an essential source for real time and short term weather
forecast”
Radar data is primarily available from national met offices, which operate
radar networks. Some companies offer to install local radars, but these
have limited coverage. The main types of data available are:
21
How do weather experts improve the data
to create a forecast?
Weather experts will typically take basic radar data, such as location and
intensity, and enhance it, using their expertise and insights to create a
unique value set of data points. For example, they use the movement of
precipitation from radar to create a movement field. This information
is then applied to real-time data to help map the possible rainfall in the
next three hours. Additionally, they’ll bring together data from multiple
sources to provide insight over a large area, such as a whole country or
even the whole of Europe.
#3 Expected radar
This improvement uses the data to create a movement field of the
precipitation, which is applied to real-time data to extrapolate the
expected precipitation for the next three hours. This type of insight
forecasting is essential for any business operations affected by
precipitation.
22
#5 Cumulative radar:
This combines multiple radar images to calculate the estimated
precipitation amount over 1-, 3-, 6-, and 24-hour periods. In some
locations, this can also be calibrated with ground observations.
23
Category One:
Weather Observation Networks
Weather
Satellites
24
Weather satellites monitor the conditions of the atmosphere, clouds, and
the Earth’s surface. Images are taken either by the infrared spectrum,
which allows cloud coverage to be observed at all time; or by the
visible spectrum, which requires daylight but provides a more realistic
visualization.
For Europe, Africa, and the Indian Ocean: Images in both spectral
wavelengths from Eumetsat’s Meteosat satellites.
For North and South America and the Pacific Ocean: Images from
NOAA’s two satellites – GOES-East and GOES-West.
25
Data is enhanced with Images from the polar orbiting satellites
(including Metop-A, Metop-B (from Eumetsat), NOAA-19, Aqua and Terra
(from NOAA and NASA).) and also derived satellite data. This including
cloud analysis images, global rainfall estimates and precipitation rate
at ground level (for Europe) and sea ice edge data (for the northern
hemisphere).
When required, this data can be further enhanced with additional data
sources, such as:
• Open data, for example, the data gathered by the polar orbiting
satellites of EU’s Copernicus project (Sentinel-1, -2 and -3)
Weather experts will process the satellite data, using their own satellite
data-processing system, and combine data from multiple satellite sources
to create a global view on an hourly basis.
Alongside this, they can provide satellite data for any customer-specific,
weather satellite-related needs and offer a range of visualization
capabilities, for example:
26
• Creating smooth, custom, cloud animations with interpolated satellite
images
27
Category One:
Weather Observation Networks
Lightning
Detection
28
During a thunderstorm, every lightning strike creates electromagnetic
waves that travel through the atmosphere at the speed of light. Ground-
based (terrestrial) antenna networks can detect these waves. Regional
networks play a vital role in accurately identifying lightning with
terrestrial systems.
29
What data is available from lightning
detection?
“High accuracy lightning data together with weather radar images allow us
to track thunderstorms and issue timely warnings for our customers.”
Weather experts will purchase and process data from a range of sources,
depending on location and need:
For Europe: The Nowcast (Ubimet) operated Linet system has the
highest detection efficiency available at up to 99% and boasts location
accuracy of up to 100m. Siemens operates BLIDS. UK Met Office operates
SFUK.
Global Coverage: Earth Networks provide TLN data (this offers high
accuracy for both types of lightning strike, focusing on North America,
Brazil, Australia, Southeast Asia, and Europe) and GLN data (delivers
cloud-to-ground detection only, with a detection efficiency between 60
and 95% and a location accuracy between 300 and 2,000m).
30
As well as using the data to identify current and forecasted weather
events, they also use it to measure and record annual lightning density
(the number of strikes per area) and to analyze historical lightning data
to consult and support their customers.
31
Category Two:
METEOROLOGICAL
AND
OCEANOGRAPHIC
(METOCEAN)
MODELS
32
Meteorological and oceanographic models (also known as metocean
models) help metocean experts to forecast the conditions in the
atmosphere and ocean over the coming hours, days, and weeks.
• Oceanographic models, for example, the spectral wave model, use wind,
ice and current, and bathymetric data to predict waves on each cell of the
grid.
33
What information do metocean models
provide?
• Oceanographic models, for example, the spectral wave model, use wind,
ice and current, and bathymetric data to predict waves on each cell of the
grid
By using the coordinate system to map the earth onto a grid, metocean
models can be used by the experts to provide:
34
• Microscale modeling: uses a feature resolving grid that models specific
environmental properties that strongly affect forecasted parameters
which aren’t in regional grids such as islands, sandbanks, and valleys.
Resolution and physics are always limiting factors with the above, but
reliable forecast results can be achieved by assimilating observations via
the global grid.
Despite this, and with the right assistance, that data can be made to work
very well. For instance, wave spectra from the ECWMF are only available
on a three- or six-hour basis, while an experienced company can take the
raw data and use it to produce data that is available for every hour (or
more frequently).
Metocean models are reliable, affordable and with the right dataset
they can provide businesses with unique selling points regarding their
forecasting abilities.
35
Category Two:
Meteorological and Oceanographic (Metocean) Models
External Metocean
Models: What’s
Available in the
Market
36
The weather today tells you a lot about the weather tomorrow. Using
meteorological and oceanographic (metocean) models, experts can
analyze current weather conditions and historical data to predict future
conditions.
“The data we receive from our global and regional weather suppliers is
included in all of the work we do and embedded in all of the in-house
models we run. The data is key to our business and we are thankful all of the
innovation in the market. Different external models have different strengths
depending on region, forecast time-step and type of weather. Our experts and
modelers use these insights to get the most value out of the data and input we
collect.”
#1 ECMWF
ECMWF is both a research institute and also a 24/7 service that produces
numerical weather models. The most important ECMWF models for
weather experts are:
37
• Atmospheric Model high resolution 10-day forecast (HRES)
#2 UKMO
The UK Met Office supplies historical data, forecast data, and specialist
datasets, providing both atmospheric and oceanographic datasets.
38
#3 NCEP
• NCEP GFS
• NCEP Ensemble
• NOAA WW3
39
How metocean experts add value to the
model data
But the data from the models alone is not enough. Working with a
specialist Weather Company, means you can get advice on how the model
data should be used and, importantly, how it can help you avoid bad
decisions made using misinterpreted model output data. A metocean
modeling company will offer several key benefits, including:
But the real value comes in helping companies understand the likely
future weather conditions and how these conditions will impact on
operations and projects. Access to accurate and complete data, with
customizable coverage, is essential to mitigate risk. By providing the
best quality datasets, assessed against both station observation networks
and end-customer observation data, metocean experts support their
customers to get more value from the data.
40
Category Two:
Meteorological and Oceanographic (Metocean) Models
In-house Metocean
Modeling: What You
Need to Know
41
External datasets, which are readily available in the market, are a
valuable part of the forecaster’s toolkit. However, they are just part of
the puzzle. For example, wave spectra from ECWMF are available every
3 to 6 hours, which might not be the required temporal resolution for
the specific use case. In-house models can provide this higher resolution
with insights for every hour - or even intra-hourly. The in-house models
can also provide spatial resolution down to hundreds of meters.
“Modelling is both an art and a science. Where science delivers the empirical
formula that form the basis of the models themselves, it is up to the metocean
modeler to simplify the complex world into an optimal configuration that
ensures maximum quality while using as little resources as possible”
It is clear that for some use-cases, external datasets alone are not
enough. They need additional inputs to solve the challenge that the
customer is facing. Custom model configurations allow experts to select
source terms (physical equations) and grid resolutions for that particular
use case. To translate global data to your specific area of interest, the
experts nest one or more feature-resolving grids in regional grids
and then the regional grids in the global grids. We call this physical
downscaling, an alternative to statistical downscaling.
In-house models provide end users with direct access to new and
improved methodologies and give them the chance to participate in
the research and implementation of future models in ways that can
specifically benefit their business.
Working with the experts, who know the models and your business,
ensures that the guidance and advice you receive are tailored to your
situation and requirements. You can focus on your priorities, confident
that you have access to near real-time insight into how the weather
conditions are developing and experts that understand what this data
means for you.
42
How do the experts create a numerical
metocean model?
At an offshore wind farm location off the Belgian coast, sandbanks can
cause high or long waves to break early, or induce waves to bend.
43
Think about offshore companies that need to plan operations in marginal
weather conditions. They benefit from specific inhouse models and
combinations. In this case, a SWAN model run on a high-resolution
grid can take spectral wave data from the regional WAVEWATCHIII,
surface winds from WRF, tidal data from harmonic components, and
ocean circulation data from Mercator, in order to properly capture, for
instance, the wave-current interactions over complex seafloor features.
The resulting dataset can provide unique insights in reigning and future
conditions. These types of modeling is also crucial for innovative blue
energy developments that depend on accurate tide and wave power data.
In-house models are built for specific use cases or for specific locations.
They aim to increase the forecast value parameters of specific data
points, for example data points relating to waves for near shore work or
specific offshore activities like cable laying.
It’s about supporting the end client at all stages of their work, with the
data that they need. For offshore projects, for example, this typically
means using coarse data for the tender phase, with detailed studies used
for design phase and then using the same model grids for the operation
and decommissioning phases.
In-house models provide end users with direct access to new and
improved methodologies, and may even give them the chance to
participate in the research and implementation of future models in ways
that can specifically benefit their business.
Working with the experts, who know the models and your business,
ensures that the guidance and advice you receive are tailored to your
situation and requirements. You can focus on your priorities, confident
that you have access to near real-time insight into how the weather
conditions are developing and experts that understand what this data
means for you.
44
Category Three:
STATISTICAL POST
PROCESSING
45
Statistical post processing is a technique used by weather experts to
enhance and improve their forecasts. It’s an umbrella term, describing
multiple statistical methodologies, which each have a unique purpose
and application.
46
Each location has its own precise local characteristics which will not be
fully reflected in a coarse-scale model. The MOS realigns the deviation
and ensures that the appropriate weighting and characteristics are
applied from the three main models. As such, a specific individual MOS
forecast is generated for each MOS observation point.
How it is used
MOS provides data for data feeds and APIs, which are used for a wide
range of applications, including crop forecasting in the agriculture sector
and reporting on extreme weather for insurance companies. It provides
input for more specialist systems, like Road & Route models. It also
underpins specific weather products, such as Dynamic Line Rating.
For critical marine operations, the weather experts also provide derived
elements like risk wind speed and risk wave height. The output data is
refreshed 4 times each day.
How it is used
NMB is specifically for marine applications in shipping and offshore.
It feeds into specialist products like SPOS and NowcastingPro. It
also underpins custom route advice for master mariners and custom
Metocean reports.
47
#3 Scalable Downsizing (ScaDo)
The weather model data used for forecasting can cover large grid areas
of up to 50km. As a result, it doesn’t always consider the impact of
features that fall within those grid lines, such as changes in elevation
and land-use. Downscaling helps to overcome this problem. During the
process of downscaling, data from the large grid areas is enhanced with
geo-statistical methods to make predictions about weather and climate
‘on-the-spot’, particularly at the surface level.
How it is used
ScaDo is invaluable for locations that experience highly localized weather
effects, such as an Alpine valley. It provides detailed wind forecasts for
wind farms, feeds into the route based forecast model and can run along
a predefined list of points like a powerline.
48
How it is used
The Road & Route models are used to identify situations where the
weather poses a risk on the roads, supporting gritting decisions. The
automotive industry also uses them as inputs for autonomous driving
initiatives.
For the route forecast model, they apply scaling to the physical model to
improve the spatial resolution; for example, by adding information about
the temperature and condition of road surfaces between the RWIS sites.
49
Category Three:
Statistical Post Processing
Model Output
Statistics (MOS):
What is it & How it
Contributes to Your
Forecast
50
Model Output Statistics (MOS) is a real showpiece at the heart of creating
your forecasts. The weather experts use it to add value to purchased raw
weather data, by correcting model data for local influences and get closer
to the actual situation.
The MOS takes account of two years of historical data and compares
local observations with the model data issued. Each location has its
own precise local characteristics which will not be fully reflected in a
coarse-scale model. The MOS realigns the deviation and ensures that
the appropriate weighting and characteristics are applied from the three
main models. As such, a specific individual MOS forecast is generated for
every weather parameter for every observation point.
“Our MOS system is the core of our worldwide forecasting system. Not only
because it adds a lot of quality to raw-model data, also MeteoGroup’s MOS
system provides the customer with specific derived weather parameters that
strongly increase the usage options”
- Floris Bijlsma,
Senior Meteorological Researcher
There are separate MOS equations relating model output with observed
weather elements (temperature, wind etc.) for every observation station
and forecast step. These equations are updated every year using up to 2
years’ worth of historical (model and observation) data.
51
Step 2: The MOS prepares the ‘main’ and ‘derived’ MOS
elements
Using the historical data it has been trained with, as well as live model
data and live observations from weather stations, radars and satellites, it
runs the MOS equations, the results of which are referred to as the ‘main’
MOS elements. It also creates ‘derived’ MOS elements; for example,
where there are no observations available, it derives the forecast from
related data. Derived data also includes probability forecasts (risk
of extreme events) and a translation of the forecast into the most
significant weather type.
Every hour, the MOS forecast is updated for the coming 48 hours and 4
times per day the MG MOS updates the forecast for the coming 15 days.
In total, MOS does billions of calculations per hour.
The combined output of the main and derived elements is “packaged” for
shipping to internal or customer systems.
52
How the MOS adds value to weather data
to create a more accurate forecast
• Using satellites, the expected cloud pattern for the next 3 hours can be
corrected.
53
How do the weather experts keep the
MOS accurate?
The weather experts play a vital role in putting the finishing touch to
the ultimate weather forecasts to be issued. In addition to the technical
and physical setup, as highlighted in Step 6 in the process to create
MOS, experienced forecasters are essential to finessing the MOS via the
MeteoBase system.
For example, when using a specific weather model mix, the MOS
may always add two degrees to a specific area to arrive at the correct
maximum temperature. However, if the purchased models are adapted
to more closely approximate the correct maximum value, adding those
two degrees as a correction factor will result in the temperature forecast
being too high.
At its core, the MOS and MeteoBase data are used to create your weather
forecast and help identify situations where the weather poses a risk.
Data from the MOS underpins other statistical methodologies like Road
& Route models, for road surface temperatures. Equally, the MOS can be
applied to create custom solutions for specific weather-related problems.
For example, de-icing operations and dynamic line rating.
54
Category Three:
Statistical Post Processing
55
Nautical MeteoBase (NMB) is a forecast data engine that feeds all
marine-related products (both shipping and offshore). NMB draws on
various sources of atmospheric and oceanographic model forecast data.
For critical marine operations, the weather experts also provide derived
elements like risk wind speed and risk wave height. The output data is
refreshed 4 times each day.
56
The weights are variable depending on the forecast lead time.
Through model mixing, NMB enhances the strong points of the input
models and reduces the weak sides, which improves the accuracy and
reliability of the forecast.
This allows the filter to learn and apply corrections on-the-fly, reducing
the mean error (bias). Because of the dynamic character of the filter, it
can adapt to changes in measurements (e.g. seasonal shifts) or changes/
upgrades in the model used.
The weather experts can apply a Kalman filter to the significant wave
height forecast produced by the Nautical MeteoBase for offshore clients.
This requires a live observation data stream between the client and the
experts. To use the filter’s full potential, the stream should deliver new
measurements (via FTP) at a high-frequency stream every 10-60 minutes.
57
How is the Kalman filter applied to waves
First, a check is done to establish whether NMB forecast data for the
client location is stored in the database.
If a live data feed is not available, this needs to be set up by the client.
If it has already been up and running, the experts will work on the
automatic ingestion of this data into the database.
Step 3: Training
The training period ensures that the Kalman filter coefficients, which
calibrate the NMB forecast, are stabilized. Once that’s done, the filter can
become operational.
The Kalman filter uses up-to-date measurements, taken just before a new
NMB forecast is released. Because of this approach, the filter can further
improve the short-range wave height forecast.
Experience shows that coastal locations benefit most from the significant
reduction in the mean error of the model forecast. These locations are
sometimes not well represented within the NMB, due to factors such as
sea-floor levels, currents, and tides. When tides and currents dominate
the working conditions for a client in a coastal (shallow water) area,
deploying a SWAN domain instead of a Kalman filter is also a serious
option, although both work well.
58
The approach taken to create a detailed and bespoke forecast will
ultimately depend on the specific requirements of each client. Working
with weather experts, companies can ensure they have access to the right
data for their needs.
59
Category Three:
Statistical Post Processing
How Scalable
Downscaling
Contributes to Your
Weather Forecast
60
The weather model data the experts use for forecasting can cover large
grid areas of up to 50km. As a result, it doesn’t always consider the
impact of features that fall within those grid lines, such as changes in
elevation and land-use. Downscaling helps to overcome this problem—
during the process of downscaling, data from the large grid areas is
enhanced with geostatistical methods to make predictions about weather
and climate ‘on-the-spot’, particularly at the surface level.
61
various elevations and certain distances from the site in question, so they
have to assign all of these observations a certain weight. This process is
called interpolation.
For our Alpine mountain top, for example, you would receive an
algorithm that forecasts the maximum temperature by assigning weights
to the surrounding points: A (30%), B (20%), and C (50%). If our site is
at an elevation of 3,000 meters, and the surrounding points are located
at least 500 meters lower, then we will also have to apply general
meteorological algorithms to arrive at a reliable forecast. The innovation
primarily lies in the increased flexibility of the algorithms, because
ScaDo takes better account of the continually changing, specific weather
conditions in the region. Because ScaDo is trained for the whole region,
it can also provide a forecast for a location without nearby weather
stations.
62
• MOS, by nature, scores better on point verification metrics like the
mean absolute error.
• Each form of processing needs to be analyzed for its fitness for purpose
and the cost versus quality of the output.
63
Category Three:
Statistical Post Processing
The Straightforward
Guide to Road &
Route Models
64
During winter road maintenance, operations staff are under pressure
to guarantee traffic safety, while maximizing operational efficiency.
Treating roads too late can lead to accidents, traffic jams, and potential
fatalities. However, unnecessary treatment of roads can result in high
costs and avoidable environmental damage.
Road & Route models are used to identify situations where the weather
poses a risk on the roads, including supporting gritting decisions and,
increasingly, supporting autonomous vehicle initiatives. They have been
specifically designed by the weather experts to address these challenges.
Here’s how the two models work:
For the road surface model, the experts use statistical modelling to
improve the physical model data with 3 years of historical observations.
This enables the road surface model to take into account the local
environment of an RWIS station. This improves the model’s accuracy.
They also run a bias filter that looks at the past 21 days to identify any
inconsistencies.
For the route forecast model, they apply scaling to the physical model
to improve the quality; for example, by adding information about the
temperature and condition at RWIS sites.
65
“ Our road and route models are built and fine-tuned based on years of
experience in working for the road weather industry.“
Through this insight, they can minimize gritting costs and environmental
impact, by avoiding unnecessary treatments and using the weather
forecast to know the right amount of salt and chemicals to use.
As well, the models can help to plan resources and improve operations,
by using the road weather forecasts to plan for personnel, chemicals, and
maintenance of equipment.
Road & Route models are not just used for road maintenance; they’re
also critical for the automotive industry and, in particular, the drive
towards autonomous vehicles.
The weather experts are working closely with some of the world’s most
prestigious car brands to come up with in-car systems that will enable
autonomous vehicles to anticipate and handle even the most severe and
most rapidly-changing road conditions.
66
Some car manufacturers are already delivering systems to help drivers
navigate away from rain and hail and towards blue skies and sunshine if
the driver decides to follow the directions. But in the future, autonomous
vehicles will rely fully on detailed, accurate, and real-time road and
weather information to ensure safety.
67
Three Ingredients to Help Put Strategic
Weather Data Into Practice
To use weather data strategically, raw weather data is only part of the
solution. Making sense of the weather for your business also relies on
knowing historical weather data and also forecasting the future. In short,
the more you know about weather patterns, the more strategic your
decision making will be. To put strategic weather data into practice, here
are the three ingredients you need.
The more you know about weather patterns, the easier it becomes to
make the right strategic decisions. Hindcasting and forecasting alone
bring you nothing but raw data, meaning you’ll need to combine this
information with other business data. With the right calculation models,
hindcasting and forecasting can be turned into concrete business
answers.
69
#3 Weather experts that understand
weather and your business challenges
Do you switch off when you hear words like “algorithms” and “models”?
Don’t worry. Meteorology is a specialism, meaning specialists that
can do the work for you. Whether you have in-house meteorologists or
not, calling in the help of a third party is a good idea when starting with
strategic weather data. This is because the possibilities are endless, and
there’s no need for you to explore them all. Sometimes, all you need is a
“yes” or “no” or a percentage, such as in the case of dynamic line rating.
Therefore, sit down with a party that knows how to use weather data
strategically and helps you deploy it IT-wise, so you can focus on results
instead of weather maps and forecasts. They can collaborate with
your own meteorologists and come up with a plan and pass on their
knowledge to them. This way, you can have your strategic weather plan
up and running in no time.
70
Category Four:
QUALITY
CONTROL & DATA
PROVISIONING
71
Weather forecasting is not something you have; it’s something you do.
Just like playing football and learning to drive a Formula 1 car, you have
to work on your skills to become better and more successful in the future.
This is important if you’re Lionel Messi or Max Verstappen, but also if
you’re a professional meteorologist.
Every day, new weather data comes in that has to be checked for
accuracy, completeness, and irregularities. New and better calculation
models not only improve weather forecasting, but also improve the
accuracy of historical data, which is equally as important to make
predictions. Clearly, quality control and data provisioning are worth the
time. But what do the weather experts check? And how do they ensure
it’s accurate?
“Our clients make decisions based on our data. These decisions can affect
the safety of people and involve high costs. At MeteoGroup, we are always
aware of the high responsibility that comes with our crucial role in the
processes of clients. Providing highest accuracy for those weather situations
where our clients and their downstream activities are most vulnerable is key
for us. When we measure data accuracy and forecast quality we take a strict
user perspective. We apply scores which are scientifically sound and still
intuitively understandable.”
Let’s start by answering a very important question: why are the continual
quality and data management improvements so important? First of all,
the difference between somewhat accurate data and highly accurate data
can make or break a business. For example, weather routing gets ships
from A to B on the optimal route.
72
Quality checks can be divided into two categories: control for
observations and control for weather models and forecasting system
output. Below, we’ll talk you through both.
73
Supporting quality checks with data
provisioning
74
Category Four:
Quality Control & Data Provisioning
How Data
Provisioning Adds
Value to Weather
Data
75
Your organization is one-of-a-kind. The data you’re using, tracking and
storing is likely to be unique to you and your needs. But over time, data
integrity can be compromised, which is bad news for business. Poor
quality data costs $3.1 trillion a year in the US alone.
The good news for you is that, when it comes to your weather data, you
don’t need to worry - so long as the experts manage it.
“The Data Provisioning team is taking care of processing weather data to the
highest standards, to give the weather experts the most accurate and reliable
information they need.”
However, the senior software engineers do much more than just solve
problems. They streamline data processes, construct the transition of all
data processes into the cloud, and extract all kinds of derived weather
data from the incoming information, ensuring that it’s ready for the
weather room.
The data is processed by the team to the highest standards, to give the
weather experts the most accurate and reliable information they need.
The data is shared with internal departments, to form the backbone of
each and every forecast.
76
The Two Data Provisioning Disciplines
77
But there’s still more to be done. Historically, data was processed via local
servers. This meant data processing had to be done step-by-step because
server capacity was limited. Essentially, when data came in, the first step
in the process could start, and only when that was finished could the next
step begins.
78
Category Four:
Quality Control & Data Provisioning
The Non-
Meteorologist’s
Guide to Weather
Data Quality Control
79
If the weather is important to your business, you’re going to care about
the quality of your data. Every element of the weather forecasting process
is rigorously and objectively monitored by the experts to improve data
accuracy and your confidence in the data. They use verification methods
recommended by the World Meteorology Organization to ensure
standards are maintained.
Evelyn Müller,
Verification Manager
Quality control is a key step for all elements of the weather forecasting
process. It’s a way of maintaining standards, by testing the outputs
against the expected output. It typically takes place in two stages:
80
How the experts apply quality controls to
weather models
The experts draw data from multiple weather models to improve the
accuracy of their forecast systems. This includes monitoring performance
and establishing a baseline for model verification, which is the on-
demand analysis of forecast against observation data for specific sites,
periods, and element. They also measure the impact of their proprietary
forecasting systems against the baseline. Here is how they apply the
checks to forecasting systems:
• Internal reports on monthly forecast KPIs for the board, which forms
the quality retrospectives.
• MOS statistics itself are also used, for example the value of any weather
parameter averaged over all stations should not change much when a
new MOS forecast is ready
• Daily verification for a subset of buoys and platforms in the North Sea.
81
Scalable Downsizing (ScaDo): ScaDo is a forecasting system for areas
with limited observation data and, therefore, there are few verification
options. To compensate, the weather experts apply an “out of sample”
verification process called ScoreCard during the development of the
forecasting method. Once ScaDo is operational, they apply a similar
method to a small set of weather stations.
Road & Route models: The forecasts are archived in the Verification
Database for:
82
Category Five:
METEOROLOGY
AND FORECASTING
EXPERTISE
83
Some people think that to come to accurate weather forecasting, you
need the right team with the right equipment. They’re not wrong.
However, in reality, you need a little more than that. Weather forecasting
revolves around data, which means that you need specialists to gather
the data, decode it, turn it into understandable information and create
customer-specific solutions that make a real difference to the strategic
decision-making process. One person alone cannot do everything that’s
needed. Or, at least, they can’t if you want to do it right. In this chapter,
we tell you about the five (!) teams that together form the ultimate
weather forecasting squad.
84
Category Five:
Meteorology and Forecasting Expertise
The Ultimate
Weather
Forecasting Squad -
5 Teams You Need
85
Team 1. Data provisioning
First up is the data provisioning team. Here is where you find data
analysts that document all incoming weather data – ingesting and
processing it in a way that other teams of the squad can understand.
The members of the data processing team speak the required technical
languages and know how to decode weather data, no matter where it
comes from. They also know how to interpret weather model changes and
changes in radar and satellite data; valuable information that they’ll pass
on to other teams.
86
Team 4. Professional services
And there you have it: five teams that together form the ultimate
weather forecasting squad. Don’t be scared though; you don’t have to
hire thirty new employees. You probably work with data analyst already,
and maybe you even have in-house weather forecasters. The professional
services team consists of people that you can “borrow”, which means you
can hire experts to help you integrate weather forecasts in your strategic
decision-making process. Weather people are flexible that way!
87
Category Five:
Meteorology and Forecasting Expertise
Improving Weather
Data One Industry at
a Time
88
While we ramble on about weather forecasting (which we hope you
appreciate), you’re perhaps starting to wonder what this means for
you. When it comes to the weather, different industries have different
challenges and different needs. If you work for a media company, for
example, you need people that can explain the weather in a human way.
If you work in shipping or offshore, you’re probably more focused on how
weather, like waves and squalls, impacts your day-to-day operations.
Sadly, high quality data doesn’t come on a silver platter (if only that were
true!). This means that all incoming weather data needs to be enriched
before it can add value in any given sector.
The work done by the weather experts across all industries can be
categorized into two disciplines: the meteorological researchers and
forecasters. The first specializes in researching and developing products,
which ingest quality-controlled data, for specific industry use-cases.
The latter is in direct contact with customers and interprets the models
for the specific customer use cases. For example, working with local
authorities to know whether to grit a road or not during the winter
months; or with offshore companies to know whether or not they can
work safely in the weather conditions.
89
8 industries = 8 approaches
#1. Media
#2. Transport
#3. Offshore
90
#4. Shipping
#5. Energy
The same goes for energy. Here, 80% is spent on meteorological research.
Sub-seasonal forecasting and dynamic line rating are important in this
industry, both of which benefits from historical data and hindcasting.
The other 20% of the work revolves around text writing, consultancy,
explanation, client-specific data enrichment, reporting, and yes: sub-
seasonal forecasting.
#6. Insurance
#7. Agriculture
91
#8. Consumer apps
92
Conclusion
THE 16 WEATHER
OBSERVATION AND
FORECASTING
VALUE
PARAMETERS,
EXPLAINED
93
If your forecast isn’t reliable, you’ll not be able to access the insights you
need to make informed business decisions. Inaccurate weather forecasts
lead to offshore companies missing weather windows during projects,
insurance companies that are insufficiently staffed during severe
weather, and transmission system operators running power lines over or
under capacity - to name just a few examples.
But how do the weather experts know when good is actually good? And
how can you recognize when the weather data you receive is held to
the highest standard? Enter into the ring the weather observation and
forecasting value parameters.
“In a probabilistic forecast you cannot take one case and say the forecast
is right or wrong. You will have to take many, many forecasts and then
compare.... Suppose 92 percent of the minimum temperatures forecasted
were correct and our competitor says: ‘we had 95 percent correct’. It is not
that plain and simple. Maybe they took a boundary of 2.5 degrees, where we
took 2.0 degrees. Well, in that case, you cannot compare the results. We are
very open with customers when we are in direct contact with them. However,
if we would just put numbers on our website, they could be misinterpreted
and misused.”
94
They are applied (as applicable) to the Five Categories that are essential
to a high-quality professional forecasting service. Let’s explore the
parameters in more detail and share how they add value to every
category.
How the experts put it into practice: The weather experts invest in
higher-frequency updates from weather stations for specific locations.
How the experts put it into practice: The experts can use an algorithm
to reduce the 5-minute interval between radar images to 1-minute.
Definition: The density of the grid (radar, satellite, and model), as well
as granularity of the weather station network
How the experts put it into practice: Standard grid sizes are 10km,
25km or 50km but the density can be increased to hyper-local or
downscaled for weather station using an algorithm. In-house metocean
models can have a spatial resolution down to hundreds of meters.
95
Value Parameter 4: Coverage
How the experts put it into practice: The experts provide global
coverage through weather stations observations, with access to a network
of 20,000 stations. Global coverage is also available for models including
MOS, Nautical MeteoBase, and Road models. ScaDo complements this, by
providing forecasts for locations with no observations.
How the experts put it into practice: All MOS forecast locations have
hourly forecasts for all elements, with downscaling algorithms used
to compensate when a station is unable to deliver data on a particular
element.
How the experts put it into practice: The in-house metocean models
provide the ability to access output parameters that are inaccessible in
external data sources (i.e., spectral moments, effective cloud cover).
96
Value Parameter 7: Accuracy
How the experts put it into practice: 98% of the forecasts are within
a 2-degree margin. The MOS provides high-quality forecasts for many
stations, while ScaDo improves the temperature forecasts in valleys and
mountains.
How the experts put it into practice: The forecasts follow the correct
pattern, even if there is a bias (structural over- or under-forecasting).
Two years of training data ensure the MOS can be adjusted to local
observation sites.
How the experts put it into practice: The experts measure whether the
forecast is a hit (correct forecast), miss (exceeding the defined threshold
but not forecasted), or false alarm (not exceeding the defined threshold,
but forecasted). The MOS combines data from several models to improve
the skill, as it takes out inconsistencies.
How the experts put it into practice: The experts apply sharpness by
ensuring the specificity of the forecast. It is the difference between it is
going to rain versus it is going to rain at 10am in Amsterdam.
97
Value Parameter 11: Uncertainty
How the experts put it into practice: Forecasters edit the MOS forecast
after comparing it with new observations and new model data. And,
when it comes to data management, the experts carry out observation
decoding, weather model changes, radar/satellite changes.
How the experts put it into practice: The experts provide a redundant
network, with 99.9% data availability. The migration to the AWS cloud,
and continuous monitoring of servers and services by the MG Operations
Center, further supports this value parameter.
98
Value Parameter 14: Availability
How the experts put it into practice: The formats include SI (Système
international) / WMO-approved units (e.g. Celsius) and industry standard
data formats.
How the experts put it into practice: More than 90% of data is
ingested, processed, and delivered within a few minutes of being
provided by a third party.
How the experts put it into practice: The experts visualize the data
and prepare it for online and on-screen presentations, which improves its
usability.
99
CONCLUSION
100
The same weather conditions can mean very different things to different
companies in different industries. Whether you want to reduce costs and
your environmental footprint in shipping, increase food production in
greenhouses, or save lives by making winter roads safer, knowing what
the weather will be, and, more importantly, how it will affect you, relies
on the right information.
Getting the weather right isn’t easy or cheap. We all have weather apps
on our phones. But this isn’t the same as accurate weather data that
support your specific needs. Accurate forecasts require substantial
investment. As we’ve demonstrated through this Ultimate Guide,
you need to manage and process the data, bring together the right
experts, and build the infrastructure to support all this work – not a
straightforward task.
About MeteoGroup
Want to know more about the way we work and how we can help you?
Get in touch!
https://www.meteogroup.com/contact-us
101
Glossary
38 WEATHER
FORECASTING
TERMS YOU NEED
TO KNOW
102
There are so many unique terms used in modern weather forecasting
that it can feel like the weather experts have their own language. To help
you get to grips with the terminology, we’ve curated together 38 weather
forecasting terms that you need to know.
Bathymetry
This is usually referenced as Lowest Astronomical Tide (LAT) or Mean
Sea Level (MSL) and refers to seafloor elevation.
Decluttering
This is a cleaning process that removes false signals or ‘clutter’ from raw
data, thus improving its accuracy. Our own algorithm can detect and
remove false signals from real radar echoes.
Downscaling
Weather model data used for forecasting is usually available on a coarse
grid and doesn’t always resolve local weather within those grid lines.
With downscaling, you can combine geospatial data with data from the
coarse grid.
ECMWF
Stands for the European Centre for Medium-range Weather Forecasting
(including the ocean).
Ensemble forecasting
Minor changes in the atmosphere can have a dramatic effect on forecasts.
Ensemble forecasting is when a weather model is run several times in
parallel with small changes to the initial conditions each time, resulting
in an ensemble or set of forecasts.
103
ENS plume
This refers to the fifty slightly perturbed runs of ECMWF atmospheric
weather model’s moderate resolution forecast. The data is used to
provide a distribution of possible values per weather element. Wider
plumes indicate more uncertain weather.
Forecast window/depth
This is the length of a forecast. For instance, a 10-day forecast window
means a forecast of ten days ahead.
Funneling
When wind parcels need to squeeze through smaller openings. This
speeds them up and is otherwise known as the Venturi Effect.
Gap filling
See interpolation.
Grid
Refers to the collection of points in two or three dimensions with
attributes describing the static environment (such as depth, elevation,
land mask, and vegetation).
Governing equations
Describe how the values of unknown variables within a model change
when one or more of the known variables changes. For instance,
conservation of mass, energy, momentum (Navier-Stokes).
Harmonic components
The gravitational pull of the sun and the moon influences the tide on
earth. These orbits are sometimes referred to as bathymetry, and it’s
possible to compute water elevation and the resulting currents via
superposition of a series of cosines.
Hindcast
This is a model study that predicts values for a time in the past. Used
mainly to compare model data with measurements to assess model
behavior.
“On-the-spot”
It’s possible to make weather forecasts hyperlocal by zooming in on
expected conditions at precise locations and point in time.
104
Interpolation
Sometimes referred to as ‘gap filling’, this is the process of filling
in the unknown data between two known values. This is usually
done by drawing on a wide range of knowledge sources from expert
meteorologists.
Metocean
A combination of the words ‘meteorology’ and ‘oceanography’. Refers to
data which originates from either field.
MeteoBase
This is the final, edited version of MOS forecasts (see MOS).
Model
A set of governing equations and utilities to input and output data.
MOS
Stands for ‘model output statistics’ and describes the post-processing
technique that reduces systematic errors in raw weather model data.
MOS station
Typically a location for which there is observation data available. It
can also be a location with no observation data available, but which is
downscaled. A MOS station can be physical or virtual.
Multi-model forecasting
Describes the process of using several weather models to generate a
(usually more accurate) weather forecast.
NCAR
National Centre of Atmospheric Research, Boulder, Colorado.
NCEP
National Centre of Environmental Prediction, a subsidiary of NOAA
located in Washington D.C.
NOAA
National Oceanographic and Atmospheric Administration, Silver Sprint,
Maryland.
Nowcasting
This describer the forecasting of weather for the short term (typically
from 2-6 hours). It can usually forecast accurately for small events such
as individual showers and thunderstorms.
105
Ocean circulation
Refers to the global current in the ocean basins that are caused by
differences in temperature, salinity, and wind shear.
Orography
The study of the topographic relief of mountains. Can also include hills
and most elevated terrains.
Residual current
See ocean circulation
Spatial resolution
The granularity or density of geographic grids that are being used to track
atmospheric conditions. Smaller grids offer higher spatial resolution and
more accurate results.
SWAN
Stands for ‘Simulating Waves Nearshore’, and is a third-generation wave
model that computes random, short-crested, wind-generated waves in
coastal regions and inland waters.
Temporal resolution
Refers to the timespan between measurements for forecasts. More
frequent measurements result in higher temporal resolutions.
Verification
Undertaken on demand, this is the analysis of forecast against
observation data for specific sites, periods and elements.
Verification database
The archive that contains raw data for the ECMWF model, MOS,
observations, maritime and road data, and more.
WRF
Stands for ‘Weather Research and Forecasting’ and refers to the model
developed by the National Centre of Atmospheric Research (NCAR).
Weather code
A number that classifies the weather conditions. For instance, fog
patches or shower with hail. They help define the weather icons we
typically see on TV forecasts.
Wx
A morse code word for ‘weather’ (.-- -..-).
106
To find out more visit our Knowledge Base
The Internet of Everything Working in the Margins: How Maximizing Efficiency and
& the Weather: A business Climatology and Metocean Data Safety Through Optimized
success formula Can Transform Offshore Project Weather Routing
107
CONTRIBUTIONS
Thanks to everyone who helped in the
making of this guide
108
Wim van den Berg
Senior meteorological consultant | Weather Systems Team
Weather Stations, Statistical Post Processing, Scalable Downscaling
Sander Hulst
Senior Oceanographic Researcher
In-house Metocean models
Dr Hugo Hartmann
Senior Meteo Scientist
Metocean models
Floris Bijlsma
Senior Meteorological Researcher
Weather Radar, MG MOS & MeteoBase
Evelyn Müller
Verification Manager
Quality Control & Data
Dennis Schulz
Chief Meteorology Officer
Meteorological Expertise
Marcel Wokke
Meteorological researcher
Road & Route Models
109
110