Gartner: 85% of AI Implementations Will Fail By 2022

A series of 2019 predictions by Gartner were reported on by Venture Beat on June 28, 2021. As explained in a prior post, “AI”, or machine learning, relies on datasets and algorithms. If the data is imperfect or incomplete, a computer has a chance of giving bad output. If the algorithm that tells the computer what to do with the data is imperfect, the computer has a chance of giving bad output. It’s easy to point to anecdotal cases where “AI” makes a bad call. There have been reports of discrimination in facial recognition technology, driverless cars killing people, or Amazon’s algorithm deciding to fire drivers that are doing their job. I’ve seen plenty of data on the failings of overhyped technology and commercial ASR. What I hadn’t seen prior to today was somebody willing to put a number on the percentage of AI solutions that succeed. Today, we have that number, and it’s an abysmal 15%.

Perhaps this will not come as a surprise to my readers, considering prior reports that automatic speech recognition (ASR), an example of machine learning, is only 25 to 80 percent accurate depending on who’s speaking. But it will certainly come as a surprise to investors and companies that are dumping money into these technologies. Now there’s a hard number to consider. And that 15% itself is misleading. It’s a snapshot of the total number of implementations, not just ASR. ASR comprises a percentage of the total number of implementations out there. And it’s so bad that some blogs are starting to claim word error rate isn’t really that important.

Judge,
I know I botched 20 percent of the words.
But word error rate really isn’t that important.

That 15% is also misleading in that it’s talking about solutions that are implemented successfully. It is not talking about implementations that provide a positive return on investment (ROI). So imagine having to go to investors and say “our AI product was implemented with 100% success, but there’s still no money in this.”

The Venture Beat article goes on to describe several ways to make AI implementation a success, and I think it’s worth examining them briefly here.

  1. Customizing a solution for each environment. No doubt that modeling a solution for every single business individually is bound to make that solution more successful, but it’s also going to take more staff and money. This would be almost like every court reporting company having their own personal software development staff to build their own CaseCAT or Eclipse. Why don’t they do that? It’s hopelessly expensive.
  2. Using a robust and scalable platform. The word robust doesn’t really mean anything in this context. Scalability is tied to modular design — the ability to swap out parts of the program that don’t work for specific situations. For this, you need somebody bright and forward thinking. They have to have the capability to design something that can be modified to handle situations they may not even be aware exist. With the average software engineer commanding in the ballpark of $90,000 a year and the best of them making over $1 million a year, it’s hopelessly expensive.
  3. Staying on course once in production. This involves reevaluating and sticking with something that may appear to be dysfunctional. This would be almost like the court reporter coming to the job, botching the transcript, and the client going “yes, I think I’ll use that guy again so that I can get a fuller picture of my operational needs.” It’s a customer service nightmare.
  4. Adding new AI use cases over time. Piggybacking on number 3, who is going to want to continue to use AI solutions to patch what the first solution fails to address? This is basically asking businesspeople to trust that it will all work out while they burn money and spend lots of time putting out the fire. It’s a customer service nightmare.

I really respect Venture Beat trying to keep positive about AI in business, even if it’s a hopelessly expensive customer service nightmare.

With some mirth, I have to point out to those in the field that believe the stenographer shortage is an insurmountable problem that we now know machine learning in the business world has a failure rate that’s right up there with stenographic education’s failure rate. Beyond the potential of exploiting digital reporters or stealing investor money, what makes this path preferable to the one that has worked for the last hundred years? As I wrote a week ago, the competition is going to wise up. Stenographic court reporters are the sustainable business model in this field, and to continue to pretend otherwise is nothing short of fraud.

Aggressive Marketing — Growth or Flailing?

During our Court Reporting & Captioning Week 2021 there were a couple of press releases and some press releases dressed up as journalism all about digital recording, automatic speech recognition, and its accuracy and viability. There’s actually a lesson to be learned from businesses that continually promise without any regard for reality, so that’s what I’ll focus on today. I’ll start with this statement. We have a big, vibrant field of students and professionals where everyone that is actually involved in it, from the smallest one-woman reporting armies to the corporate giants, says technology will not replace the stenographic court reporter. Then we have the tech players who continuously talk about how their tech is 99 percent accurate, but can’t be bothered to sell it to us, and whose brilliant plan is to record and transcribe the testimony, something stenographers figured out how to do decades ago.

Steno students are out there getting a million views and worldwide audiences…
And Chris Day? He’s posting memes on the internet.

You know the formula. First we’ll compare this to an exaggerated event outside the industry, and then we’ll tie it right into our world. So let’s breeze briefly over Fyre Festival. To put it in very simple terms, Fyre Festival was an event where the CEO overpromised, underdelivered, and played “hide the ball” until the bitter end. Customers were lied to. Investors were lied to. Staff and construction members were lied to. It was a corporate fiasco propped up by disinformation, investor money, and cash flow games that ended with the CEO in prison and a whole lot of people owed a whole lot of money that they will, in all likelihood, never get paid. It was the story of a relative newcomer to the industry of music festivals saying they’d do it bigger and better. Sound familiar?

As for relative newcomers in the legal transcription or court reporting business, take your pick. Even ones that have been incorporated for a couple of decades really aren’t that impressive when you start holding up the magnifying glass. Take, for example, VIQ Solutions and its many subsidiaries:

I promise to explain if you promise to keep reading.

VIQ apparently trades OTC so it gives us a rare glimpse of financial information that we don’t get with a lot of private companies. Right off the bat, we can see some interesting stuff. $8 million in revenue with a negative net income and a positive cash flow. Positive cash flow means the money they have on hand is going up. Negative income means the company is losing money. How does a company lose money but continue to have cash on hand grow? Creditors and investors. When you see money coming in while the company is taking losses, it generally means that the company is borrowing the money or getting more cash from investors/shareholders. A company can continue on this way for as long as money keeps coming in. Companies can also use tricks similar to price dumping, and charge one client or project an excessive amount in order to fund losses on other projects. The amazing thing is that most companies won’t light up the same way Fyre did, they’ll just declare bankruptcy and move on. There’s not going to be a big “gotcha” parade or reckoning where anyone admits that stenographic court reporting is by far the superior business model.

This is juxtaposed against a situation where, for the individual stenographic reporter, you’re kind of stuck making whatever you make. If things go badly, bankruptcy is an option, but there’s never really an option to borrow money or receive investor money for decades while you figure it out. Seeing all these ostensible giants enter the field can be a bit intimidating or confusing. But any time you see these staggering tech reveals wrapped up in a paid-for press release, I urge you to remember Fyre, remember VIQ, and remember that no matter what that revenue or cash flow looks like, you may not have access to the information that would tell you how the company is really doing.

This also leads to a very bright future for steno entrepreneurs. As we learn the game, we can pass it along to each other. When Stenovate landed its first big investor, I talked about that. Court reporting and its attached services, in the way we know them and love them, are an extremely stable, winning investment. Think about it. Many of us, when we begin down this road, spend up to $2,000 on a student machine and up to $9,000 on a professional machine and software. That $11,000 sinkhole, coupled with student loan debt, grows into stable, positive income. So what’s stopping any stenographic court reporting firm from getting out there and educating investors on our field? The time and drive to do it. Maybe for some people, they just haven’t had that idea yet. But that’s where we’re headed. I have little doubt that if we compete, we will win. But we have to get people in that mindset. So if you know somebody with that entrepreneurial spirit, maybe pass them this post and get them thinking about whether they’d like to seek investors to grow their firm and reach. Business 101 is that a dollar today is more valuable than a dollar tomorrow. That means our field can be extremely attractive to value investors and be a safe haven from the gambling money being supplied to “tech’s” habitual promisors.

Know a great reporting or captioning firm that needs a spotlight? Feel free to write me or comment about them below. I’ll start us off. Steno Captions, LLC launched off recently without doing the investor dance. That’s the kind of promise this field has. I wish them a lot of luck and success in managing clients and training writers.

Turning Omissions Into Opportunity

We’re in an interesting time. Pretty much anywhere you look there are job postings for digital reporters, articles with headlines talking about our replacement, articles with headlines talking about our angst. Over time, brilliant articles from people like Eric Allen, Ana Fatima Costa, Angie Starbuck (bar version), and Stanley Sakai start to get buried or appear dated when, in actuality, not much has changed at all. They’re super relevant and on point. Unfortunately, at least for the time being, we’re going to have to use our professional sense, think critically, and keep spreading the truth about ourselves and the tech we use.

One way to do that critical thinking is to look squarely at what is presented and notice what goes unmentioned. For example, look back at my first link. Searching for digital reporting work, ambiguous “freelance” postings come up, meaning stenographer jobs are actually branded as “digital” jobs. District courts seeking a stenographer? Labeled as a digital job. News reporters to report news about court? Labeled as a digital job. No wonder there’s a shortage, we’re just labeling everything the same way and expecting people who haven’t spent four decades in this business to figure it out. In this particular instance, Zip Recruiter proudly told me there were about 20 digital court reporter jobs in New York, but in actuality about 90 percent were mislabeled.

Another way to do it is to look at contradictions in a general narrative. For example, we say steno is integrity. So there was an article from Lisa Dees that shot back and said, basically, any method can have integrity. Can’t argue there. Integrity is kind of an individual thing. But to get to the conclusion these things are equal, you have to ignore a lot of stuff that anyone who’s been working in the field a while knows. Stenography has a longer history and a stronger culture. With AAERT pulling in maybe 20 percent of what NCRA does on the regular, who has more money going into ethics education? Most likely stenographers. When you multiply the number of people that have to work on a transcript, you’re multiplying the risk of one of those people not having integrity. We’re also ignoring how digital proponents like US Legal have no problem going into a courtroom and arguing that they shouldn’t be regulated like court reporters because they don’t supply court reporting services. Even further down the road of integrity, we know from other digital proponents that stenography is the gold standard (thanks, Stenograph) and that the master plan for digital proponents is to use a workforce that is not highly trained. I will totally concede that these things are all from “different” sources, but they all point to each other as de facto experts in the field and sit on each other’s boards and panels. It’s very clear there’s mutual interest. So, again, look at the contradictions. “The integrity of every method is equal, but stenography is the gold standard, but we are going to use a workforce with less training.” What?

Let’s get to how to talk about this stuff, and for that, I’m going to leave an example here. I do follow the court reporting stuff that gets published by Legaltech News. There’s one news reporter, Victoria Hudgins, who has touched on steno and court reporting a few times. I feel her information is coming mostly from the digital proponents, so in an effort to provide more information, I wrote:

“Hi Ms. Hudgins. My name’s Christopher Day. I’m a stenographer in New York. I follow with great interest and admiration most of your articles related to court reporting in Legal Tech News [sic]. But I am writing today to let you know that many of the things being represented to you by these companies appear false or misleading. In the August 24 article about Stenograph’s logo, the Stenograph offices that you were given are, as best I can tell, a stock photo. In the September 11 article about court reporter angst, Livne, says our field has not been digitized, but that’s simply not true. Court reporter equipment has been digital for decades. The stenotype picture you got from Mr. Rando is quite an old model and most of us do not use those anymore. I’m happy to send you a picture of a newer model, or share evidence for any of my statements in this communication.

Our position is being misrepresented very much. We are not worried so much about the technology, we are more worried that people will believe the technology is ready for prime time and replace us with it without realizing that it is not. Livne kind of admitted this himself. In his series A funding, he or Verbit stated that the tech was 99 percent accurate. In the series B funding he said Verbit would not get rid of the human element. These two statements don’t seem very compatible.

How come when these companies are selling their ASR, it’s “99 percent” or “ready to disrupt the market,” but when Stanford studied ASR it was, at best, 80 percent accurate?

Ultimately, if the ASR isn’t up to the task, these are transcription companies. They know that if they continue to use the buzzwords, you’ll continue to publish them, and that will draw them more investors.

I am happy to be a resource on stenographic court reporting technology, its efficiency, and at least a few of the things that have been done to address the shortage. Please feel free to reach out.”

To be very fair, because of the limitations of the website submission form, she didn’t get any of the links. But, you know, I think this stands as a decent example of how to address news people when they pick up stories about us. They just don’t know. They only know what they’re told or how things look. There will be some responsibility on our part to share our years of experience and knowledge if we want fair representation in media. It’s the Pygmalion effect at work. Expectations can impact reality. That’s why these narratives exist, and that is why a countering narrative is so important. Think about it. When digital first came it was all about how it was allegedly cheaper. When that turned out not to be true, it became a call for stenographers to just see the writing on the wall and acknowledge there is a shortage and that there is nothing we can do about it. Now that’s turning out not to be true, we’re doing a lot about it, and all we have left is to let those outside the industry know the truth.

Addendum:

A reader reminded me that Eric Allen’s article is now in archive. The text may be found here. For context purposes, it came amid a series of articles by Steve Townsend, and is an excellent example of what I’m talking about in terms of getting the truth out there.

What Verbit Investors Need To Know

I had touched pretty gently on Verbit when its series A funding came in at $23 million. The series B funding is in at about $31 million earlier this year. Now Verbit’s announced a strategic partnership with the STI and professional flip flopper, Jim Cudahy. Migliore & Associates already came out with the hard truth of what this means: ASR doesn’t make the cut for the production of legal transcripts without a qualified court reporter no matter what you name it, NLP, ASR, AI, computer magic, automated transcription.

Do I come off as angry? I am angry. I’m angry that investors are being led down a path of burning capital where there’s just not a bright future. When the series A funding was happening, Verbit used words like automated, “save an enormous amount of manual labor.” “Adaptive speech recognition” with over 99 percent accuracy. Series B is out. They “would not take the human transcriber out,” “the AI will enhance the human.” So investors are fundamentally paying millions of dollars so that they can be another Rev. I doubt very much that that’s what was sold to investors. I don’t think anybody would be putting down millions on that.

Then the partnership with STI? A complete joke. I have already gone into how, without any doubt, stenographers and NCRA are by far the best equipped to deal with the court reporter shortage. AAERT and the STI just don’t have the funding, infrastructure, or experience to tackle the problem, and it shows in their data. By their estimates, court reporting companies stand to save $250,000 over the next decade by adopting digital tools. First, I would love to know if this is individual savings or cumulative. We don’t know because there are no sources linked or cited. If this is cumulative, it’s embarrassing that they would even post that. That would mean 25,000 in savings a year across all companies. If that’s the projected individual savings per company, only slightly less embarrassing. That’s less than the average annual salary of a single court reporter. This may come as a shock to Jim Cudahy, but court reporting companies adopted digital tools throughout digital’s birth in the 70s and into the 80s and 90s. Stenographers are already a part of the Information Age, utilize AI, and produce quality records daily. The idea that investors are going to dump $50 million into “technology” expected to save $250,000 over 10 years and expect a return is terrifying. “Most courts are digital,” again, assuming everything they have to say is true, and yet judicial candidates show a preference for stenographic court reporters and returning them to courtrooms. The growth here is in stenographers, stenography jobs, and stenography schools, and Verbit’s current leadership is missing this boat completely.

Let’s just tell it like it is. When a grassroots-funded stenography blog can give you some pretty solid reasons you’re backing the wrong horse, it’s time to give investors nothing less than what they deserve. Open up a Steno Department, throw down some money on us, and we will make sure you’ve got real and steady returns. Verbit, with proper leadership from Tom Livne, can still save the day. Just not with this bait and switch technology-to-transcription model that amounts to little more than a repackaging of old tech. The only other viable alternative I see is buying this blog for a good $8 million and hoping investors don’t see it before then. Not a difficult decision. Come on over to the winning team. Vote for sten!

July 18, 2021 Addendum:

I reread this article for the first time in a while and I feel I did an inadequate job of explaining just how pathetic the cost savings promoted were. I took to social media to spread the word, and I am copying and pasting what I had to say below.

“I’m slowly reconstructing the table of contents on my blog, and I came across an old article I did on Verbit. In that article, it mentioned a blurb by STTI that, using AAERT info, estimated $250,000 in savings over a decade for companies that switched to digital tools.
The reason I’m bringing it up again is this is a time where a lot of court reporters are kind of concerned. “Is this stuff better than I am?” And I pointed out in the article how I think some $50 million was sunk into Verbit at that point. $250,000 over a decade breaks down to $25,000 a year. So even if it’s assumed that that information is completely factual and that the savings is for every individual company in the field and NOT cumulative across the industry, they were advertising savings equivalent to less than the average salary of a single court reporter.
Is this stuff better than you? They need to sink millions of dollars into it to “maybe perhaps” save $25,000 a year. Remember, these are companies that boast millions in revenue. I think you have your answer.
And for anybody that doesn’t, I have a handy chart here.
If they make $1 million, they save 2.50%
If they make $2 million, they save 1.25%
If they make $3 million, they save 0.83%
If they make $4 million, they save 0.63%
If they make $5 million, they save 0.50%
If they make $6 million, they save 0.42%
If they make $7 million, they save 0.36%
If they make $8 million, they save 0.31%
To put this into perspective, a company like VIQ Solutions, parent of Net Transcripts, that posts something like $8 million a quarter (not profitable), would save 0.08%.
A company like Veritext, which Owler claims has revenue in the $400 million ballpark, would save 0.007%
Just to put some more perspective on that, if Stenograph were to offer you 0.007% savings on the purchase of approximately $4,000 CaseCAT, you would save $0.28. ($100 at 2.50%)
Would you pay anyone millions of dollars to “maybe perhaps” save 0.007% to 2.50%?
This situation is very good for us, but we do need more reporters, so in this coming school year, if you can make a visit to your local high school, please do.”

Veritext Update, March 2019

Introduction & History.

First and foremost: This post is going to get into past history and then go into more recent history. In the more recent history, in order to prove that what we’re saying is true, there are screen shots of a person’s LinkedIn social media. We’re free to discuss that and we’re free to say how we feel, but any reader that comes here should be aware that harassment, bullying, menacing, stalking, and defamation are all amoral and illegal. Those things may all open you up to criminal and civil action. If you use our steno news as a gateway for antisocial behavior, do not be surprised if you get police at your door.

Now onto history. Veritext was a leader in working to bolster stenography. A quick Google search will show you that assuming all the media out there to be true or partially true, they are a partner to NCRA and do or did, on some level, and sometimes on an astounding level, support the stenographic methodology for taking the record. It is hard to tell if what follows is a case of Yes, Prime Minister’s advice on backstabbing or a case of the principle of hedging. Veritext proceeded to buy out a lot of stenographic or court reporting companies, including Diamond Reporting here in New York. Next, we caught wind that Veritext was advertising to attorneys that they should change their deposition notices to add language of “stenographic or other means”, presumably so that Veritext could choose to send digital reporters to jobs.

This all ended up culminating in a post where we mirrored SoCalReporter’s ideas and said: We need to stop beating each other up about where we work and start talking solutions. Guess what happened? People started coming up with solutions, and content, and even going so far as to create watchdog groups. We have said this before, but we are seeing a memetic shift. The reporting zeitgeist of silence is over. There are hundreds of voices blogging, talking, and working together to come up with new ideas.

Today & Tomorrow.

So that brings us to the end of February 2019. A woman named Gina Hardin, purportedly a VP of Sales at Veritext, wrote or posted an article about digital reporting being the changing landscape of reporting. There was a great deal of chatter about this, culminating in the post being taken down the night it was posted, and an immediate declaration from Veritext that the post was posted by a former employee and that they had nothing to do with it, honest. This doesn’t pass the colloquial “sniff test” or SMELL test for being true. Why would a former employee try to drum up business for a past employer? In this country, with so few rights for workers, what employee would ever go out on a limb and post something like that without their employer’s explicit permission? Unless you work for the government or have a contract saying otherwise, you can be fired for any reason or no reason, even a made up reason, just not an illegal reason, of which there are very few. The whole thing just doesn’t make sense. And if she’s a former employee, apparently nobody told her, because as of March 2, 2019, she was still listed as working at Veritext, but under the name Gina H. It’s all but undeniable that Veritext is pushing digital, including hiring via their website.

Now, here’s the deal: Some people went online and talked about the typos in the article, or even had personal attacks. It’s not about her. As best we can tell, she’s an employee doing a job, and probably doing it damn well. We make a thousand typos a day unless we’re Super Stenographer. Stenographers, and the entrepreneurs among us, should really be looking at teaming up with salespeople like that who’re dedicated to their job and willing to put themselves out there. Though we have not yet gotten a chance to interview Eve Barrett of Expedite Legal, one of the things she’s alluded to online is there’s an amazing power in human-to-human marketing because of this digital, faceless world. Who is going to be better at human-to-human marketing than someone who is willing to attach their face to the product and pitch? We wouldn’t be surprised if there are stenographic companies looking to poach Gina H. or salespeople like her right now! There’s huge money in this field. Nearly every big agency has a satellite office in every borough of New York City and a cadre of dedicated employees — in other words, there is money to be made in this field, and we shouldn’t be afraid to hire talent when it means a bigger return. Success is often a matter of intelligent delegation. As stenographers, we often let our penchant for perfectionism stand in the way of hiring help and building our brand, perhaps to a fault.

But where does that leave us? Well, we need to recognize that Veritext is apparently willing to lie. Freelancers need to recognize that group boycotts by competitors may fall under antitrust violations. Reporters everywhere need to start acknowledging that the best way to beat ’em might be to just start grabbing clients. It’s time for us to get serious about funding our associations and demanding marketing and entrepreneurial courses. These companies all exist because they got clients off of somebody else. Individually, they may seem bigger or stronger than us because they can outspend us one-on-one, but there’s an inherent power in the fact that if thousands of reporters were to compete directly with them and start poaching clients — which is perfectly legal unless you signed a contract saying you wouldn’t do that or stole a trade secret — they’d be SOL.

For the most ambitious, start looking at fundraising. Start considering all the ways companies come into existence. You very well could be the next nationwide conglomerate. As a matter of fact, if you’re in Illinois, New York, California, or Texas, you are in one of the largest court reporting states in the country, and you have a real shot at seizing the market. Companies rise and fall — but your career is in your hands.

We look forward to the day Veritext sees it’s on the losing side and starts throwing its weight behind stenography again. We look forward to dutifully reporting that right here on this blog. But until that day comes, we encourage fierce competition in this market. Don’t be complacent. Maybe someday we’ll get SLAPP’d for standing up for our profession, but we’re happy to take the heat so that you don’t have to. Be involved. Encourage others to get involved and start building their brand. Know that you are making a difference in how the market and our day-to-day jobs develop.