Generative AI – The New, Shiny Bright Object: Play with This Tool, Do So Cautiously

Generative AI – The New, Shiny Bright Object: Play with This Tool, Do So Cautiously

CEIR’s Nancy Drapeau shares her experience so far using generative AI in her research including the good, bad and the ugly.

By Nancy Drapeau, IPC, CEIR Vice President of Research

The benefits and threats of Artificial Intelligence have been hotly debated as of late among opinion leaders in technology, academia, public policy and am sure at family dinner tables and dinner parties across the globe. As AI technological advancements happen at an ever more rapid pace, experts are concerned they present more harm than good, possibly an existential threat to humanity. Yikes!

I don’t have that message for you in this blog. Instead, I want to urge you to jump in and play with Chat GPT, or other AI tools such as Google Bard, Bing Chat, among others. But I also urge that you do so cautiously. Do not adopt these tools and set aside current practices. Instead, think of how they can support your activities, rather than replace them.

Why am I urging caution? Technological advancements that satisfy the unquenchable thirst for convenience have been a major engine for innovation. They get adopted, mainstreamed, maybe sometimes faster than is wise. Think of driverless cars. Chat GPT and AI technologies will enjoy adoption where they can reliably accomplish tasks faster, more efficiently, less expensively. Though today, they are not completely proven to consistently generate accurate results. So, user beware.

Am always fascinated that people are quick to place high confidence in the output generated by machines. There seems to be more confidence in machines than people sometimes! It’s as though it is assumed that output from machines is superior to human output. Machines need to prove their superiority. They have for certain purposes, such as cars overtaking horse for transportation, airplanes over trains, phone calls/text messaging over walking over to someone’s office, etc.

However, generative AI is a black box of machine learning tools – algorithms, neural network analysis, etc. I do not profess to be an expert in machine learning technology. Though I do recall, when assessing multivariate analysis systems my employer was considering purchasing, neural networks were an analytic approach available to conduct certain analyses. And I remember being told that how results are generated via neural networks is a ‘black box.’ It isn’t known how the results are arrived at. That gave me pause. Now that was over 10 years ago. A well-known neural network is the Google search algorithm, so am hoping there is greater command of ‘how’ results are generated today. All the same, when using generative AI tools, make sure you can replicate the approach, to assure quality assurance of the results it generates.

Some uses are quite straightforward, having the tool write draft copy for an email or advertisement does not present risks. The output is what it is. The user can then take that draft copy and tweak it, to make it better. However, using it for other purposes, such as relying on the results of analyzing content for research is a riskier endeavor. It begs the question, how reliable is it?  

To-date, Chat GPT AI is relatively new, it is a work in progress. Output has been uncovered that is inaccurate, completely wrong, and that the system may ‘hallucinate.’ That might be entertaining though go slowly in using the tool for business purposes. Take it for what it is, do not over rely on it.

I have a Chat GPT account. I have used it to support text analysis of open-ended comments to surveys we have run this year. Here is my assessment of how the tool performed:

  • It is a great starting place to develop overarching themes articulated in open-ended questions.
  • At the same time, it misses some categories/insights, it fails on capturing certain detail that provides context to what is said.
  • Text analysis software has been around for decades, the more sophisticated systems enable a user to train the system to capture relevant contexts (sentiment – what is happy vs. sad) and terms (industry jargon) that a general system might not catch. Chat GPT is apt to miss this kind of detail.
  • And quite honestly, I wasn’t clear on why it picked up on some themes and not others. There appeared to be some arbitrariness to the process. Yes, I have high expectations for Chat GPT output generated in seconds!
  • Though again, I used it to get a flavor of what was being said in the commentary, and then created code frames to dig deeper.
  • At last year’s Predict, one of the guest lecturers, Tricia Wang, who calls herself a digital ethnographer warns professionals to not over rely on quantitative data (data providing insights on a population with statistically reliable results, or big data generated by the likes of Google, etc.) and to make sure to pay attention to ‘thick data’ or what researchers call ‘qualitative data’, those anecdotal insights that might be diamonds in the rough, insights that may point to an important trend to consider. She advocates doing analyses that capture both quantitative and qualitative insights. I agree. The Chat GPT tool, in its current format – is choppy, inconsistent, and apt to miss key, qualitative insights. Something to keep in mind as you use the tool.

Last thing, let me summarize key takeaways from an  Industry Insights Association webinar I listened to last week entitled, Generative AI in Insights: Innovation, Opportunity, Risk & Regulation. The session was moderated by Insights Association CEO Melanie Courtright, panelists included: Stuart Pardau, IA General Counsel, Jamin Brazil, Managing Director, HubUX (Voxpopme), Howard Fienberg, SVP, Advocacy, Insight Association; Cynthia Harris, Managing Director, 8:28 Insights and Roddy Knowles, Senior Director, Research and Product-Led Growth, DISQO

Here are key takeaways and some further commentary from me as well:

  • Begin using AI Generative tools such as Chat GPT, Bard or other like tools. Play with the tool regularly, for at least 30 minutes a week.
  • Do not use generative AI as a replacement for research. It is best used as a starting place. It can support a variety of tasks, such as creating survey questions, conducting text analysis (my experience), doing preliminary discovery research, etc. To apply this to applications in the B2B exhibition industry, it can assist in writing copy for promotions, for website content, emails, etc. Though again, use it as a STARTING place. Edit, refine, make it better!
  • Generative AI is bad at prediction. Researchers need to be the curators of what it provides.
  • Not getting started using Chat GPT is a mistake. Don’t wait for permission, the establishment of company regulations on its use.
  • However, there are risks. Assure use of the tool is in line with your company’s privacy agreements. Careful on use of personally identifiable information. Make sure content is not infringing on existing copyrights.
  • Technology always outpaces regulations. We are seeing that here. Laws passed, interpretation of them, case law, will take time to catch up to this new technology. The European Union is likely to lead in generating regulations, as they led efforts that resulted in GDPR.
  • Do not copy and paste proprietary information into the Chat GPT window. That content will be absorbed into the machine learning process. Here is how to exclude your chat output from Chat GPT machine learning: https://openai.com/blog/new-ways-to-manage-your-data-in-chatgpt Though note new chat content is retained for 30 days before it is deleted. Basically, once you have gotten the analysis you want, copy and paste that content OUTSIDE of Chat GPT, such as in Word and Excel and then click on the trash icon to purge it from your Chat GPT account.

Yes, it is a brave new world. Though go forward with courage, play with the tool. Use it cautiously…

Other articles of interest:

https://www.pewresearch.org/internet/2021/06/16/1-worries-about-developments-in-ai/
https://www.nytimes.com/2023/05/30/technology/ai-threat-warning.html#:~:text=A%20group%20of%20industry%20leaders,with%20pandemics%20and%20nuclear%20wars.

https://www.pwc.com/us/en/tech-effect/ai-analytics/managing-generative-ai-risks.html

About the Author

 

Nancy Drapeau, IPC. A 29-year market research veteran with more than 20 years in the business-to-business exhibitions industry. As CEIR’s Vice President of Research, Ms. Drapeau conducts industry wide studies and reports on current trends in the exhibition industry. In 2019, she was named to BizBash’s 1,000 Most Influential People in Events list. She holds a BA in Government from Georgetown University and a Master’s in Advanced European and International Studies from l’Institut Européen des Hautes Études Internationales. She is an AC Nielsen Burke Institute trained focus group moderator. She is a well-respected industry speaker and an active member of the Industry Insights Association and member of the Event Industry Council’s (EIC) Research & Advocacy Task Force. She lives in Maine, with her husband, son and a border collie named Moxie.

The views and opinions expressed by blog authors are those of the authors and do not necessarily reflect the official policy or position of the Center for Exhibition Industry Research. Any content provided by our bloggers or authors are of their opinion. All content provided on this blog is for informational purposes only. CEIR makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. CEIR will not be liable for any errors or omissions in this information nor for the availability of this information.

Share this:

Related

AI Governance: The Complex Conversation We Should Be Having

5 Ways to Master AI for Business Growth Without Technical Overwhelm

The Top 10 Cities in 2024 by Economic Power