GM's autonomous vehicle Cruise division is already going through a rough patch, with the California Department of Motor Vehicles (DMV) recently suspending its driverless permits over safety issues. Now, several new reports have highlighted other issues with the company, including problems with its autonomous vehicles (AVs) recognizing children and the frequency with which human operators must remotely take control. The company also just announced that it's temporarily suspending production of its fully autonomous Origin transport.

The most concerning issue is that Cruise reportedly kept its vehicles on the streets even though it knew they had problems recognizing children, The Intercept reported. According to internal, previously unreported safety assessment materials, Cruises autonomous vehicles may have been unable to effectively detect children in order to take extra precautions. 

"Cruise AVs may not exercise additional care around children," the document states. Because of that, the company was concerned that its robotaxis might drive too fast near children who could move unexpectedly into the street. Cruise also lacks data around child-specific situations, like kids separating from adults, falling, riding bicycles or wearing costumes. 

In one simulation, the company couldn't rule out a scenario where a vehicle strikes a child. In another specific test drive, a vehicle detected a child-sized dummy but still struck it with a mirror at 28 MPH. The company chalked up the problems to a inadequate software and testing — specifically, it lacks AI software that could automatically detect child-shaped objects around the car and maneuver accordingly.

In a statement to The Intercept, Cruise admitted that its vehicles sometimes temporarily lost track of children by the side of the road during simulation testing. It added that the problem was fixed and only seen in testing and not on public streets, though it didn't say what specific actions it took to resolve the issue. A spokesperson also said that the system hadn't failed to detect the children, but did fail to classify them as such. 

It further stated that the odds of an accident involving children were relatively low. "We determined from observed performance on-road, the risk of the potential collision with a child could occur once every 300 million miles at fleet driving, which we have since improved upon. There have been no on-road collisions with children."

The report also notes that Cruise AVs have trouble detecting large holes in the road, such as construction site pits with crews inside, something the company itself called a "major risk." GM's own documents indicated that even with its small AV fleet, a vehicle was likely to drive into such a hole at least once a year — and into a pit with people inside once every four years. 

That scenario almost happened, according to video reviewed by The Intercept. Onboard cameras show an AV driving right to the edge of a pit, inches away from workers, despite the presence of construction cones. It only stopped because someone waved a "slow" sign in front of the windshield. 

"Enhancing our AV’s ability to detect potential hazards around construction zones has been an area of focus, and over the last several years we have conducted extensive human-supervised testing and simulations resulting in continued improvements," the company said in a statement. "These include enhanced cone detection, full avoidance of construction zones with digging or other complex operations, and immediate enablement of the AV’s Remote Assistance support/supervision by human observers."

All of that raises the question of whether Cruise should be operating its vehicles on public roads. "If you can’t see kids, it’s very hard for you to accept that not being high risk — no matter how infrequent you think it’s going to happen," Carnegie Mellon engineering professor Phil Koopman told The Intercept. 

The child detection issue isn't the only recent exposé on Cruise, as it turns out that the robotaxis aren't really autonomous at all. In fact, they require human assistance every four to five miles, according to a report in The New York Times confirmed in large part by Cruise CEO Kyle Vogt in Hacker News. 

"Cruise AVs are being remotely assisted (RA) 2-4 percent of the time on average, in complex urban environments." wrote Vogt. That equates to someone intervening every four to five miles, which could be multiple times on many trips. There is typically one remote assistant "for every 15-20 driverless AVs," Cruise stated later.

In a statement to CNBC, the company provided additional details: "Often times the AV proactively initiates these [remote assistance actions] before it is certain it will need help such as when the AV’s intended path is obstructed (e.g construction blockages or detours) or if it needs help identifying an object," a spokesperson wrote. "Remote assistance is in session about 2-4 percent of the time the AV is on the road, which is minimal, and in those cases the RA advisor is providing wayfinding intel to the AV, not controlling it remotely."

Finally, it appears that Cruise has halted production of its Origin autonomous vehicle after the California DMV pulled its license, Forbes reported. In an all-hands meeting with employees, Vogt, referring to the DMV license withdrawal, stated that "because a lot of this is in flux, we did make the decision with GM to pause production of the Origin," according to audio from the meeting. A GM spokesperson later confirmed to Engadget that "We are finishing production on a small number of pre-commercial vehicles and after that, plan to temporarily pause production."

Cruise is still operating its AVs in California, but now must have a human backup driver at the wheel. Meanwhile, California says it has given Cruise a path back to driverless operation “The DMV has provided Cruise with the steps needed to apply to reinstate its suspended permits, which the DMV will not approve until the company has fulfilled the requirements to the department’s satisfaction,” it said in a statement. 

Update, November 7 2023, 12:44PM ET: A statement from GM has been added confirming the pause.