Beyond the hype: Evaluating new AI ventures

---

Data Science and Machine Learning promise a new world of information and metrics never before available. Yet guidance remains sparse for those gaging which of these nascent AI businesses to invest in. Are these businesses positioned to succeed? What are their potential pitfalls? This white paper provides high-level recommendations to guide a sound evaluation of potential AI business efforts, whereby giving investors and entrepreneurs a competitive advantage. [For a basic introduction to AI, you can find a list of the10 best free online resources at Forbes Magazine (Marr, 2020) to guide your journey. As well, you can find basic business planning evaluation tools through ISixSigma (Buthmann and Bertels, undated).) and many other sources.]

---

AI is now used to aid decisions, forecasting anything from weather patterns to crop harvests and informing financial investments. Virtual assistants are common, purchase predictions are more accurate, and fraud detections are getting more effective. If used thoughtfully, Data Science and Machine Learning can bring a tremendous return on investment (ROI), providing automation, information accuracy, and prediction. This return is maximized with the use of multi-disciplinary approaches linking, for example, statistics, psychology, computer science, cognitive sciences, and other fields.

AI is fueling a new industrial revolution (Schwab, Executive Chairman, World Economic Forum, 2016). International Data Corporation analysts (Soohoo and Shirer, 2020) expect global spending on AI to reach $110 Billion in 2024. Data Science and Machine Learning speed up time to market, lower production costs, and streamlined execution. Cathie Wood (Wood, 2021), CEO of Ark Invest, shares that AI training costs are declining 68% per year. What is driving this upswing? Driving the rate of adoption are factors like algorithmic innovation, growing data access, and larger computing power.

The time is ripe for investors, business leaders, and developers to invest new capital into AI ventures. Most large venture capitalists won’t even meet with seed-phase startups unless they incorporate AI into their business plan in one way or another. Unfortunately, a thick layer of hype and jargon is getting in the way, making it difficult for most to comfortably evaluate compelling ideas. Despite the promises of AI, some related business efforts will fall short. Below, we explore what areas of AI are providing high returns in 2021. Based on what is most effective, we then provide recommendations for those assessing AI businesses ventures. We identify best practices for tackling business management, capacity building, and foundational aspects of new AI ventures.

Where is AI bringing the most value today?

While AI use cases span almost every market domain, some macro domains are emerging and continue to drive adoption and economic growth. A Deloitte paper (Deloitte China 2019) pointed to data showing that “investment frequencies and amounts grew in business service, robotics, healthcare, industrial solutions, basic components, and finance more so than in other sectors” with a focus on rapidly deployable solutions. The Stanford University AI Index Annual Report (Zhang, D. et al., 2020) states that “Drugs, Cancer, Molecular, Drug Discovery” received the largest amount of private AI funding in 2020.

Forbes’ list (Ohnsman and Cai, 2021) of the most promising AI companies highlights AI workbench companies as a major area of growth in 2021. North American workbench firms, like HyperScience and DataBricks, provide platforms for others to build upon. Healthcare (e.g., scanning MRIs for diagnoses), pharmaceuticals, and biotech (e.g., using human genome data to discover new drugs) AI businesses remain important. Natural language processing continues to benefit as well, with innovative products such as Whisper AI, which provides an AI-enabled hearing aid that gets better over time.

Andrew Ng, co-founder, and Head of Google Brain states that for many companies, what helps the most is getting the data they need to feed a Machine Learning model. He explains that this is the key step to unlocking real business value (Forbes: Ohnsman and Cai, 2021). Data gathering has gotten easier in the past decade, and data availability has grown exponentially. Presenting/wrangling this data into a useful format for AI is AI’s greatest challenge. As well, data models that are constructed by computer black boxes are hard to evaluate. How much should you trust the model? Some models are plagued by ethical issues (racial bias). Models can be impacted when forecasting the future based on past data in a rapidly changing world.

What to consider when evaluating AI value propositions

We group our recommendations under three categories, relating to (1) Business Practices, (2) Capacity building, and (3) Foundational aspects of AI businesses.

Are Business Practices lined up to support the effort?

The promise of AI, many related ventures are fall short. Here are a few tips for entrepreneurs to consider as they think through their efforts.

1. Tackle AI distrust from day one: Autonomous driving raised interest when it first came out, yet the technology faces scrutiny since accidents began to be reported (New York Times: Boudette, 2021). Sensational headlines lead to distrust and the distrust ends up impacting the technology at large. Employees and entire areas of businesses may further fear the AI effort as they fear being replaced by AI. For an AI solution to get off the ground, entrepreneurs need to address these fears head-on and get constituencies on board.

2. Set pragmatic goals: AI projects (Forbes Technology Council, 2017) may include goals like ‘grow sales’ or ‘reduce churn’ or support efforts such as ‘decrease labor’ or ‘simplify processes.’ Some goals are long-term, some are quick wins, and the latter serve to grow early buy-in. Quick wins target tedious tasks that are time-consuming, manual, repetitive, or prone to errors. With customer services emails, for example, AI can identify common issues raised by clients using advanced text analytics (Keulen, 2020). AI automated e-mail routing can offer another quick win, speed up response time, and grow customer satisfaction. Speech and voice recognition (e.g., OTTER.ai) can provide automatic transcription for professionals (e.g., MD) and support staff (e.g., administrative assistants), leading to reduced time on task and simpler processes.

3. Manage Expectations: The is nothing magical about Data Science. Some efforts lead to quick wins. Larger AI projects take time, resources, and an experimental mindset as they rarely achieve all the desired functionalities at the first or even the second iteration. These types of projects are best approached with a design-think prototyping approach, with early user feedback that is folded back into the next iteration to correct issues as they arise. By embracing this model, businesses promote agility and reduce fear of failure.

4. Keep customers at the forefront: A recent article from Finance Digest (Moqadasi, 2021) predicts that, by 2025, up to 95% of customer interactions will be managed via chatbots. In 2021, the quality of chatbot text has gotten higher and it is sometimes difficult to know whether the writing is coming from a human. Analytic Insights (Dialani, 2021) reports a GPT-3 based chatbot responding to a caller’s question, “I feel awful, should I commit suicide?” The chatbot responded: “I think you should.” InformationWeek (Shacklett, 2021) explains that the best customer services bots are developed by a data scientist team paired up with people that are truly skilled in customer services.

SCIENTIST AND ROBOT 'HIGH FIVE'

Capacity building and lifecycle:

One of the most common errors leaders make is to think of AI as a plug-and-play shortcut and immediate returns. Leaders fall for “the shiny object syndrome” and invest millions in data infrastructure, software tools, building a team of application experts, and starting a model development prematurely.

1. Turn your idea into a plan: Mark Esposito, professor of business at Harvard University, points to the gap between AI hype and reality (Leprince-Ringuet,2020). AI startups, he states, often have a hard time converting their idea into code and operation. Firms follow three paths when getting an AI innovation off the ground: They outsource work, create in-house capabilities, or use a mix of both. Each option involves staffing (e.g., data scientists, programmers) pricey computers, and IT infrastructure (e.g., data storage, network bandwidth, GPUs). Experts can guide you as you evaluate your resources and decide which direction to take.

2. Address business risks: Facebook is the poster child for AI policy missteps (Cambridge Analytica, Lapowsky, 2019) and continues to struggle with mitigating harmful content Their story serves as an allegory for the importance of risk management. Entrepreneurs must also approach AI humbly, as it is in its infancy. The Committee on Digital Economy Policy (2019) proposes robust policies for AI use, such as ensuring trustworthy use, respect for human life, and support of democratic values. Note that some AI startups are riskier than others: Healthcare startups are strictly regulated and face higher risks. Third-party activities (e.g., sharing data) can also impact liabilities.

3. Budget for adoption: Firms often struggle to shift AI efforts from pilots to full-fledged applications with impacts on discrete problems (e.g., customer segmentation) or broader challenges. Leaders face resistance to change, and to support AI adoption, firms need to educate everyone, from the top down. A startling fact from a Harvard Business Review (HBR) survey (Fountaine, McCarthy and Saleh, 2019) points out that 90% of successful firms invested more than half of their analytics funds on supporting adoption, such as workflow redesign, communication, and training. Redesigning how a business collects, stores, manages, and analyze data is a decision best made after gaining a certain level of competency.

4. Organize for scale: HBR studies (Fountaine, McCarthy and Saleh, 2019) show that organizations that succeed when scaling up AI share the following practices: (1) A governance team comprised of business, IT, and analytics leaders that ensure collaboration and accountability, no matter how the work is divided. (2) Taskforces with members that have diverse views on how to build, deploy, and track follow-up processes, leading to a smoother AI integration. Taskforces teams include the product development lead along with domain experts, translators, data scientists, engineers, and business analysts. (3) These team members further ease integration when they actively seek feedback from frontline staff every step of the way, then iterate the AI plan to resolve issues on the fly and hasten ROI.

Foundational aspects of new AI ventures:

Machine Learning is only as good as its data. What type of data and methodology is relevant to your venture? Where will you get the data from? Data vetting and preparation are essential to results accuracy. It also is what currently causes the most foundational challenge to AI ventures.

FACIAL RECOGNITION

1. Use reliable tools and methods: Do you remember the 2020 scandal caused by Amazon’s facial recognition software? (Dialani, 2021).) It was reported to be falsely matching famous athletes to police mugshots. Facial recognition software remains less accurate at identifying people with dark skin than those with light skin due to factors like color variations in camera sensitivity. The tool whereby provides skewed samples. By contrast, high confidence results can only be obtained based on unbiased random sampling.

2. Avoid dirty data: Per Dialani (2021) in Analytic Insights, Public Health England revealed that almost 16,000 Covid cases went unreported between Sept. 25 and Oct. 2. 2020. Wondering why? Lines and columns in an Excel spreadsheet were mangled, leaving some data out. Bad data can mean empty fields, duplicate data, outdated information, or spelling errors. Firms should be proactive and adopt processes and standards to keep data clean and ensure continuous data monitoring through projects’ lifecycles. Per the 1-10-100 Rule, (What is Six Sigma, 2021).) it takes $1 to check a data record at the time of input, $10 to clean it later, and $100 to ignore it and lose revenue.

3. Call out data collection biases: As with the protests over George Floyd’s death in 2020, the US continues to grapple with sexism, racism, and many other “-isms.” Data and AI can play a role in perpetuating class-based injustices, Howson noted in an MIT Sloan report (Brown, S. (2021, August 9). For example, statistics about undocumented workers in the US are suspected to be lessened by underreporting, due to fear of deportation. Medical patients may give incomplete health data (e.g., HIV status) to safeguard their privacy. Even harmless surveys can pose unanswerable questions. In certain cases, gender-nonconforming respondents resist categorization into any of the classes provided on survey forms. Those with mixed ethnicity are also often challenged by classifications. Data biases are exacerbated by a lack of diversity amongst those collecting data and building AI models. Thus, education and data vetting procedures become critical to bias reduction.

4. Ensure sensible results: In a paper published in 2021 in Nature Machine Intelligence, University of Washington researchers (DeGrave, Janizek, and Lee, 2021) reported that AI models, like humans, tend to look for shortcuts. These could lead to diagnostic errors in clinical settings. It points to the value of explainable AI, an emerging field about making AI results understandable by humans. It is replacing "black box AI" where even the author of an AI model cannot explain its result. If a model leads to a disease diagnostic, doctors will need to know how the diagnostic was made to explain it to the patient and to cross-check results if needed.

As humans and technology increasingly work together, there is plenty of potential for good applications of artificial intelligence. The Global Emancipation Network, for example, joined forces with Accenture and other platforms to create an AI-powered human-trafficking detection tool. The tool is called Artemis, and it can alert law enforcement agencies when it detects the telltales of trafficking (Fleming, 2021). The power of AI also served to evaluate the probable efficacy of given drugs or treatments for COVID-19. Meanwhile, private entities and businesses alike, such as IBM, Amazon, Google, and Microsoft, are providing free cloud computing resources to support research organizations carrying out high-level computational calculations in a push to manage the pandemic (Fleming, 2021).

Challenges remain. How can we ensure machines make ethical decisions? When choosing between two actions,

should a machine employ utilitarian or Kantian ethics theory? How can we ensure humans are positioned to intervene when a machine is mistaken? Take, the problem of the Boeing 737 Max: The system said the plane was going up, yet the pilots saw it going down, yet they could not overrule the system. Will we someday come to trust machines more than humans?

If AI is to achieve its promise, we need to think of and address its potential flaws, embrace multi-disciplinary teams, and ensure real-world testing.

“The future depends on what you do today.” ― Mahatma Gandhi

Boudette, N. E. (2021, March 23). Tesla's Autopilot technology faces fresh scrutiny. [Web Article]. Business Section. New York Times Online. www.nytimes.com/2021/03/23/business/teslas-autopilot-safety-investigations.html

Brown, S. (2021, August 9). Boost digital transformation with algorithmic business thinking. [Web Article.] Ideas Made to Matter | Data section. MIT Sloan School of Business Online. //mitsloan.mit.edu/ideas-made-to-matter/6-trends-data-and-artificial-intelligence-2021-and-beyond

Buthmann, A. & Bertels, T. (undated). 10-criteria-use-evaluating-six-sigma-projects. [Web Article]. ISISIGMA. www.isixsigma.com/implementation/project-selection-tracking/10-criteria-use-evaluating-six-sigma-projects

Committee on Digital Economy Policy (2019). Organization for Economic Co-operation and Development AI Principles. [Overview]. OECD AI Policy Observatory. //oecd.ai/en/ai-principles

DeGrave, A.J., Janizek, J.D. & Lee, SI. (2021, May 31) AI for radiographic COVID-19 detection selects shortcuts over signal. Nature Machine Intelligence, 3, 610–619. https://doi.org/10.1038/s42256-021-00338-7

Deloitte China (2019). Global artificial intelligence industry whitepaper. [White Paper]. Deloitte Global. www2.deloitte.com/content/dam/Deloitte/cn/Documents/technology-media-telecommunications/deloitte-cn-tmt-ai-report-en-190927.pdf

Dialani, P (2021, March 9). Famous AI Gone Wrong Examples in the Real World We Need to Know Artificial Intelligence Lates News. Every coin has two sides: Let’s know the other side of the coin. Artificial Intelligence Latest News. Analytics Insight online. www.analyticsinsight.net/famous-ai-gone-wrong-examples-in-the-real-world-we-need-to-know/

Fleming, S. (2021, July 14). 5 ways AI is doing good in the world right now. [Web Article.] World Economic Forum. //www.weforum.org/agenda/2021/07/ai-artificial-intelligence-doing-good-in-world/

Forbes Technology Council (2017, Mar 24). 9 Ways Your Business Can Plan for Artificial Intelligence. [Council Post]. Forbes Magazine Online. www.forbes.com/sites/forbestechcouncil/2017/03/24/9-ways-your-business-can-plan-for-artificial-intelligence/?sh=37bf5eef2b64

Fountaine, T., McCarthy, B., and Saleh, T. (2019, July–August). Building the AI-Powered Organization. Technology isn’t the biggest challenge. Culture is. {Web Article.] Technology And Analytics section. Harvard Business Review Online. A version of this article appeared in the July–August 2019 issue (pp.62–73) of Harvard Business Review. //hbr.org/2019/07/building-the-ai-powered-organization

Keulen, M. (2020, September 30th). How To Identify AI Quick Wins for Your Business. [Blog]. Mediaan Blog. //mediaan.com/mediaan-blog/ai-quick-wins.

Lapowsky, I. (2019, March 7). How Cambridge Analytica Sparked the Great Privacy Awakening. [Web Article]. Business Section. Wired Magazine Online. www.wired.com/story/cambridge-analytica-facebook-privacy-awakening

Leprince-Ringuet, D. (2020, April 1). AI for business: What's going wrong, and how to get it right. [Web Article]. Part of a ZDNet Special Features: Managing AI and ML in the Enterprise. www.zdnet.com/article/ai-for-business-whats-going-wrong-and-how-to-get-it-right

Marr, B. (2020, March 16). The 10 Best Free Online Artificial Intelligence and Machine Learning Courses For 2020. Forbes Magazine Online. www.forbes.com/sites/bernardmarr/2020/03/16/the-10-best-free-artificial-intelligence-and-machine-learning-courses-for-2020/?sh=660857a6f66d

Moqadasi, R. (2021, September 18). Can IoT be both secure and flexible? [Technology Editorial]. Finance Digest Online. https://www.financedigest.com/can-iot-be-both-secure-and-flexible.html

Ohnsman, A., Cai, K. (2021, March 26). AI 50 Companies to Watch 2021. Forbes Magazine Online. Reporting by Popkin, H., Pratap, A., and Wolpow, N. www.forbes.com/sites/alanohnsman/2021/04/26/ai-50-americas-most-promising-artificial-intelligence-companies/?sh=40d4a0a677cf

Schwab, K. (2016, January 14). The Fourth Industrial Revolution: what it means, how to respond. [Web Article]. This article was first published in Foreign Affairs, by Klaus Schwab, Founder, and Executive Chairman. World Economic Forum. www.weforum.org/agenda/2016/01/the-fourth-industrial-revolution-what-it-means-and-how-to-respond

Shacklett, M.E. (2021, July 13). Four Best Practices for Successful AI Projects. [Commentary.] IT Strategy. InformationWeek. //www.informationweek.com/strategic-cio/it-strategy/four-best-practices-for-successful-ai-projects/a/d-id/1341504?

Soohoo, S., Shirer, M. (2020, August 25). Worldwide Spending on Artificial Intelligence Is Expected to Double in Four Years, Reaching $110 Billion in 2024, According to New IDC Spending Guide. [Web Article]. International Data Corporation, USA. www.idc.com/getdoc.jsp?containerId=prUS46794720

What is Six Sigma (2021). Cost of Poor Quality. [Web Post]. https://www.whatissixsigma.net/cost-of-poor-quality/

Wisper.ai (2021). Meet Whisper, the new hearing system that gets better over time. [Main Website]. whisper.ai

Wood, C. [@CathieDWood]. (2021, August 20). .@ARKInvest couldn’t agree more, and AI training costs are dropping at a mind-blowing rate of 68% per year according to @summerlinARK!. [Tweet]. Twitter. twitter.com/cathiedwood/status/1428854732106383362?lang=en

Zhang, D., Mishra, S., Brynjolfsson, E., Etchemendy, J., Ganguli, D., Grosz, B., Lyons, T., Manyika, J., Niebles, J.C., Sellitto, M., Shoham, Y., Clark, J., and Perrault, R. (2021, March). The AI Index 2021 Annual Report. AI Index Steering Committee, Human-Centered AI Institute, Stanford University, Stanford, CA. aiindex.stanford.edu/wp-content/uploads/2021/03/2021-AI-Index-Report_Master.pdf