Monthly Archives: September 2018

Meteorologists Need To Be Data Scientists And Programmers. Their Future Depends On It.

Posted on by
1

Data is really, really important to any business that matters. Data is being used to not just get a heartbeat on what customers are buying and using; it’s being used to drive decisions within the business. As I highlighted yesterday, companies are spending large amount of money (and their profits!) to better understand their customers’ behavior and purchasing profile.

Alternatively, meteorology requires quality data. Looking at surface, radar, satellite, and upper-level trends is a very valuable part of creating a forecast, but it will only take you so far…perhaps 12 hours into the future in some but not all situations. Computer guidance is needed to make forecasts that are longer-term. Knowing where severe storms will develop, where that hurricane will go, and how much snow will accumulate requires data and an understanding of it.

Meteorology isn’t a rewarding science unless the results are shared with many; there is little value in knowing where a storm will go and what impact it will have if only you know what the forecast is. If emergency managers, state officials, and the public know where the tornado is moving and what impacts a city will get from a tropical system, the information is far more valuable and can be used to make and inform other decisions.

Unfortunately, meteorologists can’t just download their analysis and knowledge into other people’s heads. The world would be a much smarter, better place (I hope) if we could freely exchange information and insight by installing it directly in a brain…but that isn’t the case. This requires a meteorologist to communicate their insights, conclusions, recommendations, and the data that supports all of these decisions. The conclusions, recommendations, and insights are valuable, but people and decision makers want to see the data for themselves to make sense of it.

The best meteorologists I know use quality data to tell a quality story. With so many options to get a forecast, it’s important to stand out; this unique story requires unique data. So why do so many meteorologists struggle with getting this unique data?

I want to emphasize the word “unique” here. Radar data is everywhere. Satellite data is everywhere. Model data is everywhere. Forecasts are everywhere. I’ve seen the National Hurricane Center’s track of the hurricane, and odds are good most have, too. Basically, if all or most of the weather communicators have access to or use the information, it’s not unique and people have no reason or incentive to choose one source over another.

So let’s go back to the question: why do meteorologists struggle with getting unique data? As someone that knows a fair amount about statistics, data, computers, and weather…programming is difficult. To be a successful programmer, you – among other things – must know:

  1. Where to get the data
  2. How to scrape or gather the data
  3. Format and quality control the data
  4. Know the tools to process and analyze the data
  5. Output the data or insights into a specific format

This process is not easy. There’s a lot weather data out there, and there are many meteorology-specific and non-meteorology-specific tools out there. These tools have a learning curve, and these tools take time to learn.

Secondly, finding the source of the data takes skill, time, and patience.

Let’s look at this problem. What does weather impact? How about rivers? Here’s a recent forecast for the Cape Fear River near Wilmington, South Carolina:

sep11-blog-rivers

This is a helpful plot of information. It shows past, current, and forecast observations with tide information embedded. As I write this blog, Hurricane Florence is nearing the Carolinas, so flooding and river flooding information is even more valuable. So if I’m in the weather communication world, how do I share this information with my customers? I could screengrab it and share it out, but my bosses may not be happy with NOAA’s brand all over that data.

On the page where I pulled this data, there is an XML and RSS link to the underlying data in the plot above. Within the XML and RSS link, there are observed and forecast data points and corresponding times. We can download this raw data, process it, and export it to a weather communication computer system. So how do we get the data from where is it to where we want it? The answer is: figure it out! Don’t get defeated, don’t get overwhelmed, don’t get angry; instead, research:

  1. What programming languages read and process XML and RSS data
  2. How to use those languages to write code and transform the input data into desired output data
  3. The ways to automate the process.

Next, you have to take the output and use it as an input for your weather communication computer system. That’s where your knowledge of the latter gets used to finish the project and tell a unique weather story.

What else is impacted by weather? Power outages, air quality, river levels, road conditions, and more. Those who source the data (power companies, air quality agencies, etc.) typically publish their data in some form to the Internet. So how do you get that data from their final source to your final source? The answer is: figure it out! You can do this, but you have to have the passion, energy, and time to make it happen. It’s important to note that extra time to make a unique product today may save you a lot of time in the future.

Before you think that you can’t get information from that weather-related data website, you can. If you can get it in a browser, you can get the data. You just have to try harder to get it and transform it.

The ultimate goal here, as I alluded to in my blog post from yesterday, is to create a sustainable competitive advantage and cross-promote your product. If you own product A and B, you tell people on Product A to get a unique forecast on Product B, and you tell people on Product B to get a unique forecast on Product A, you’re likely to keep your company happy while also giving your company value.

Need an example of this in action? Try this:

This WTOL-TV’s real-time school delay and closing map. That’s data from an external source (a school closing/delay crawl system or the backend where schools districts report their delays) being inputted into a broadcast machine and colorized/formatted in the machine’s software. There is no doubt in my mind there is (automated) programming here to both export and import the data from one system to the next and to convert the output of one source to an input for another system. This is impressive work, and this is the only place I know that can show this type of data to customers! This took programming, weather communication system, and local school district knowledge plus time, effort, and – perhaps most importantly – passion.

This tweet from Ryan Hoke at WAVE-TV in Louisville, Kentucky shows rainfall data from Louisville Metropolitan Sewer District rain gauges. The data is freely available here. You could import the data straight into Microsoft Excel (which is NOT a programming tool!) and export it out, but there is likely programming going on in the background here to parse the table of rainfall totals. How did Ryan made this map? He saw the challenge, and he figured it out! It’s not incredibly easy to pull this data; some websites have easy access to data, and this site is not one of them. But you can do it!

These are two (rare) examples of importing unique weather-related data into a weather communications system. The company brand is intact, and the freely (well, seemingly) available data is natively flowing into a system where data is shared. This process took a lot of time, effort, passion, patience, and perseverance…but these data can now be promoted, cross-promoted among platforms, and used in real-time to inform the masses.

But wait; there’s more. You can archive this data and leverage of the power of statistics to model out river levels, air quality numbers, and power outages. I know it can be done because I’ve done it.

As I write this blog post, Hurricane Florence is moving west towards North and South Carolina, prompting people to fill up and move inland. Here’s an impactful tweet:

https://twitter.com/GasBuddyGuy/status/1039349719464652800

If you dig hard enough on the Internet, you can find the raw data for this map and bring it into a weather communications system. What a great weather story element!

You have to work hard if you want to tell a better story than everyone else. If customers see value in your product, they will come to you to get that value. But there’s a downside: if you look like everyone else, have the same data as everyone else, and tell the same story as everyone else…you’ll have the same market share as everyone else and be as valuable as everyone else.

The ability to program and use data to drive business decisions and profitability is becoming more important every day. If you can’t leverage weather data to your advantage, your potential as a meteorologist is severely capped.

Weather Communications Has A Problem And Business Thinking Can Fix It

Posted on by
1

As a meteorologist, a weather communicator, and someone to aims to be humble, I’ve come to accept and respect that the weather community has a critical issue that needs to be addressed. As a businessperson, I see business problems being solved daily, but I also see a solution to the mounting weather communications problem.

This problem is better shown than explained. It’s worth noting that there is a lot of great weather communication out there, and I’m not going to target specific people or companies as much as the product itself. To show this example, I will use New York City, a place I have never worked and with weather communications companies for which I have never worked.

As I write this, Hurricane Florence is a major hurricane in the Atlantic Ocean. Here’s what the New York City NBC affiliate has to say recently:

When I read this, I think to myself:

“What is the ‘breakdown’ coming up? You just gave me the whole forecast. Why do I need a breakdown?”
“What’s on the app that I can’t get here? You just gave me the current conditions of the storm and earlier you tweaked out the track.”
“What are the latest updates? When are these updates coming? Didn’t you just tell me 95% of what I need to know about the approaching hurricane?”

Here’s some of what CBS in New York has to say:

The screaming message here is it will be a cloudy, wet, breezy day in New York. But I’m left with the following questions:

“I could get this on my smart phone now. Why do I need to see your communication that takes up more than a minute of my time?”
“You just told me what was going to happen today. Why do you need to tell me again?”
“There’s a lot of information about others, but I just want the information about me.”

Here’s some of what ABC in New York has to say:

Again, I’ve grabbed some tweets that showcase local warnings, a snapshot of current conditions, and a link to a website where Florence is likely heading…but I’m left with the following questions:

“When is the greatest risk here and south of here from Hurricane Florence?”
“How is Florence going to impact me?”
“You showed me it’s raining now. When is it going to stop?”
“I see the radar and the warning. Where can I go for more information?”

There are other weather communicators out there talking about New York, and here are some of those tweets:

This is just a sampling of tweets (from Twitter). People can follow and trust who they wish, people can get weather from other social media sources, they can get weather information from non-social media sources, or they can – to be honest – live their live without getting informed on the weather at all.

As someone who used to work as a meteorologist (and still is one per my degree) but now works in business, I see the communications problem more clearly now.

Among other things, successful businesses have:
– A product or service people want to buy or use
– New products and services over time to create growth
– Profitability, especially if it’s increasing
– Good leaders, ethics, goals, and boundaries
– Create unique value, especially compared to others
– Have competitive advantages, especially sustainable competitive advantages

I would argue that the last three are perhaps the most important in business. What good is a company without leadership, growth, a bright future, and a moral compass? What good is a business without giving something valuable to customers? And what good is a company that has a product or service that others can easily replicate and scale?

Corporations in this country spend BILLIONS of dollars each year to gather insights about their customers that create value and generate sustainable competitive advantages. I work at one of those companies; every day, I look to create a unique product or service – either externally or internally – that gives the company for which I work value and advantage.

I think Kroger (the grocer) is one of the smartest companies out there. They bring groceries you buy to the store, sure…but that’s not half of the value. They were among the first to give you an incentive to have a shopper’s card; they got your information and purchasing profile, and you get discounts and free items. They give you an incentive to download and use their app; they get your information and additional purchase profile data, and you get access to coupons that you don’t have to carry around the store. Kroger also sends you coupons in the mail to incentivize you to go to the store. They have ClickList that, for a fee, has someone shop for you…and you just pick up the groceries outside of the store; the value proposition here is to save time. Then there’s fuel points: you earn them when you spend money at the store. When you’ve collected enough points, you get a discount on fuel at one of their gas stations. Plus, you get extra fuel points when you buy certain items – like gift cards – in the store. They have a large selection of “store brands” in their stores to save you more over name brands.

Kroger isn’t the only one with good ideas, but they have a value stream. They know how to get you in the store, to the gas pump, using their app, and using their services. They make money not because they create value for you, but also because they have many ways to get your money and connect the value from one place they own to another.

Amazon Prime is similar. You pay for Prime and get free shipping, but they also give you free streaming services, too. Other business have big time value, but the goal here – if you’re a business – is to make money ethically and fairly in as many places as possible and have a value stream that others can’t (easily) replicate.

I tell you all of this to highlight the weather communication problem: there is very little sustainable competitive advantage in weather communication right now. ABC, NBC, and CBS in New York (above) didn’t just give the same or similar forecast; they also:

– Have the same look and color scheme
– Have no information I can’t get on a free/ad-free smartphone app, from Alexa, or on the Internet
– Largely lack a link to easily get additional valuable information
– Largely lack a call to action or impact
– Don’t cross-promote
– Don’t incentivize

The other weather tweets :

– Largely lack a link to easily get additional valuable information (or don’t motivate)
– Largely lack a call to action or impact
– Take a significant amount of time to read or process
– Lack the locality of weather
– Don’t incentivize
– Don’t cross-promote

Marketing a weather forecast has accurate or the most accurate will only take you so far when convenience and value are in play. When all horses in the race run the same, the results will be the same. All of the above groups lack differentiation or sustainable competitive advantages.

New York isn’t the only place where this is an issue, and Twitter isn’t the only place where the problem occurs. So how do we beat the lack of differentiation, sustainable competitive advantages, and incentivization? With differentiation, sustainable competitive advantages, and incentivization, of course!

Weather communicators need to create forecasts, products, and services that no one else can easily create and can cross-promote. The “face” or “person” of reliability and accuracy is dying at the successes of convenience, the discovery of a more “reliable” source (perceived or real), and the fact that seemingly “all forecasts look the same, so any weather update is fine for me.”

Whether it’s power outages, river impacts, business closings, school closings, or some other proprietary forecasts/information and a way to communicate that forecast and drive people to that forecast and drive people to other forecasts on other places within the same brand, what unique value is the weather communication community giving customers?

Meteorologists and weather forecasters need to start thinking like a businessperson. If billion dollar companies are investing billions in creating value and finding a sustainable competitive advantage…why isn’t the weather community working to figure out what their customers want and create the intellectual property to give customers something they can’t get anywhere else?