While organizations have recognized the application of natural language generation (NLG) for over a decade, it has only recently began to take advantage of the data-driven approaches guided by modern business intelligence (BI). Cale Berkey, Chief Solution Officer at Decisive Data, tells us how NLG currently fits in the BI space and shares insight into where it’s heading.

Automated Insights: When businesses evaluate NLG technology, a question that’s often asked is how to implement it. Can you talk about how your company has adopted NLG?

Berkey: Decisive Data is just beginning to embrace NLG internally. We recently redesigned our internal reporting, and our new CEO Alissa Seiple laid out new performance metrics and KPIs. We look forward to integrating Wordsmith into these reports to quicken time to insight and shorten internal performance feedback loops.

Automated Insights: So, would you say creating a new strategic approach has been key for adoption? 

Berkey: For us it has been because that approach prompted the creation of new analytics infrastructure, which naturally allows for consideration of new tools.  But I certainly wouldn’t say NLG adoption requires first the creation of a new strategic approach.  In fact I’ve seen it work the other way around.  Just this week I recommended a reference architecture to a customer targeting automated narrative report generation as the pilot his organization will use as an entrée into larger strategic analytics investments.  So I think it’s really more about the use case and value to the business – and that can be demonstrated as both a function of and as a lever into new strategy.  

Automated Insights: Tell us more how NLG impacts BI.

Berkey: NLG enhances and advances BI. Many organizations that leverage BI spend time manually producing written narratives. They want to highlight the most important analytic insights as well as reduce the time spent consuming these insights. A good example of this is producing performance narratives for executives. Self-service BI and analytics tools are incredible at exposing different areas of business performance, but in some cases that flexibility comes at a cost: it can take time to set your filters, pivot views, contextualize data, etc.  

NLG helps reduce the time spent producing and consuming analytics by automating generation of key insights in an easily digestible manner, making it a fantastic tool for organizations to expand their analytic capabilities into the executive suite. There’s a very clear ROI here for most organizations, since the cost to repeatedly and manually produce written narrative is easily calculated. Manual narrative generation also carries a severe failure mode since a human has to transcribe and check their figures.  

Automated Insights: Aside from solving the deficiencies of manual reporting, is NLG providing any unique benefits?

Berkey: Some information is just better communicated through narrative. Visual analytics are excellent at communicating descriptive, diagnostic and predictive information, but in my opinion can be less effective at communicating prescriptive information. Earlier stages of the analytic process require interpretation – what’s happening currently, what caused it, what will happen in the future – and what should I do about that? By the time you get to prescriptive, the analytics user should not only interpret what she sees but also be given explicit direction.  So we’re starting to see some organizations leverage NLG as the vehicle for serving up prescriptive information because it so naturally communicates direction. 

Automated Insights: It sounds like there’s a lot of opportunity for NLG as a prescriptive tool. 

Berkey: Absolutely. The purpose of analytics is to influence human behavior, and the market needs better solutions to help drive that influence. I can imagine a system in which sales teams are served up prescriptive narrative alongside visual analytics. BI tools know their user. If the system knows who you are and what you’re filtering – what questions you’re asking the system – it can also send back generated narrative specific to those variables. Something like, “85 percent of our teams advance opportunities like this when using the AwesomeProduct collateral, and it doesn’t look like the customer has seen that yet.” Like Clippy for analytics. Ok, Clippy doesn’t get a lot of love but still the opportunities here are endless.  

As organizations mature analytically and build up more data around behaviors and actions, it should become possible to do things like A/B test behaviors like this. Send one instruction set to some teams, another to others, and examine the results.  NLG makes it possible to set up and run experiments like this quicker. I’m very excited to see the market continue to mature in this area.

Automated Insights: You mentioned visual analytics earlier. How would you describe the relationship between NLG and data visualization today?

Berkey: What’s really exciting is that the market is sorting all this out right now. There’s a common language that’s built up around visual analytics. An average analytics user will know how to set filters and interact with most analytics dashboards and the best designers will make that process intuitive, but that common interface doesn’t exist yet relative to narrative information. What’s the best way to enhance visual analytics with narrative? Under what analytics scenarios should this be done and not done? Once it’s there, how do we know it’s influencing people the way we want?

Right now, we’re seeing great value in the automation of narrative generation. As I mentioned earlier, there’s an easy and clear ROI here for most organizations. But, I suspect the real value is something we haven’t seen really take off in the market yet and that’s the arrival of a sort of common interface – an expected standard – to work with narrative information.  

Currently we think about analytics visually. I bet in most analytics organizations if you asked, “What do you think of when you hear the word ‘analytics’,” you’ll get a lot of visual metaphors. I think NLG has a clear relationship with data visualization in that it will be used to great effect in communicating information that just isn’t well communicated via elements like charts and graphs. It will be placed alongside data visualization because that’s the common interface for a lot of organizational analytics today. But as predictive analytics expand in the market, we’ll start to see NLG contribute to prescriptive scenarios in ways that don’t or can’t include pure data visualization. I think within this context, NLG represents a sort of unique future of analytics.

Automated Insights: What else is challenging NLG?

Berkey: Organizational creativity is a significant challenge. Automation of narrative information is already a known solution and awareness is probably the only thing holding organizations back from leveraging NLG in those scenarios. NLG is much more than an automation tool. It’s a new dimension of analytics and we haven’t seen the full impact of adoption yet. Organizations that were first to adopt self-service tools, processes, and patterns built a powerful new organizational muscle. They shortened feedback loops to gain insight, built shared understanding of the realities of a business’ challenges and opportunities, and increased the speed of experimentation. However, certainly not every organization capable of building this kind of muscle has done so.  

Organizations that find creative ways to experiment with their data, push their analytics to predictive and prescriptive applications, and think first about which behaviors they want to influence should find an incredible asset in NLG. Adoption requires analytically visionary leadership and, for now at least, a comfort with ambiguity. But like those organizations early to adopt self-service analytics, forward-thinking organizations should also become competitive leaders as people gain faster and more directive insight from their analytics applications.

Automated Insights: Similarly, are there any misconceptions of NLG to address?

Berkey: A couple misconceptions I’ve seen out there are one, that NLG is a set-and-forget black box, and second, it’s “simply” a companion to visual analytics.  

Regarding the first, you still need to know your data. You still need to know your business. You still need talented analytics professionals. Organizations can’t equally measure everything that’s measurable and NLG isn’t something you simply feed data into and get told what to do. The power right now is in understanding your data and businesses and knowing what behaviors you want to influence, then crafting the narratives that will help drive behavior. NLG represents both possibility and speed in that certain analytics applications are not practically possible without NLG. How should businesses address this? Awareness is key. Organizations need analytics leaders who pay attention to whitespace and can experiment with new technologies and processes. 

To my second point, visual analytics are defined, of course, by the “visual” component. Again, organizational creativity is key here because there will need to be some sensitivity to experimentation and trying new things to best understand how to drive the right behaviors. The industry will redefine the visual interface to include narrative information as adoption increases and that will be done because forward-thinking organizations will lead the charge. 

Automated Insights: Speaking of forward-thinking, what’s next for NLG in BI?

Berkey: If the conversations I’m having are any indication, the next thing is broader adoption of automation scenarios. This will help organizations save and expand on narrative generation, and free up analytics horsepower to tackle new problems to drive the business forward.  

What I’m looking at beyond this is the development of analytics language dialect that includes narrative information. One of the things I’m most proud of  at Decisive Data is the strength of our visual artistry. I can’t wait to see our teams and customers embrace NLG further so we can express our creativity and leadership with this new analytic dimension.

Automated Insights: Let’s wrap up with a piece of advice–if you could say one thing about NLG to other companies seeking to become more data-driven, what would it be?

Berkey: Start small! In our strategic work we often recommend that organizations build pilot solutions targeting key business differentiators. This allows organizations to wrap their arms and minds around analytics and experience data-driven decision making in a way that’s both feasible and relevant. You don’t need to boil the ocean to realize the benefits of NLG. You only need to know what matters to your business. If you’re looking for some help here defining and building a pilot, give Decisive Data a call. We help organizations make better decisions with data and we can certainly help yours. 

About Decisive Data
Decisive Data is a firm of analytics professionals dedicated to world class insight for decisions, specializing in data visualization, analytics, and engineering in a multi-cloud world.