4 stars
A frightening depressingly realistic future dystopia where the U.S. has commissioned a Risk Assessment Administration (RAA) to detain people that an AI algorithm predicts are likely to create future crimes.
Sara, a Muslim working mom of young twins, returns from a conference in London to find herself detained at LAX passport control based on a prediction that she will murder her husband. This is based on an undefined black box of data including that from a Dreamsaver implant, prescribed to treat sleep deprivation but that secretly uses AI to spy on people’s dreams as a predictor of future criminal behavior. Even if you do not have the Dreamsaver device, the government keeps risk assessments on everyone, tracks biometric data, and sets a random score of 500 above which you get retained as a risk to society.
Despite calling it a retainment, Sara’s clearly imprisoned in a decommissioned elementary school with her every move watched, recorded and evaluated. Like the many other women there, she’s held long past the 21 days proscribed observation period. She’s made to do forced labor rating AI-generated content on how human-generated it seems- oh the irony in this!
Sara’s freedom and privacy get completely stripped away as she lingers in a vague, undefined evaluation in which nothing is in her control. It feels like the government version of disappearing people, and no one seems inclined to evaluate the accuracy of the algorithm that targeted her or whether it has racial profiling implications. Plus, the for-profit private company owning the retainment centers has more profits to gain by keeping people there as long as possible.
As you read the novel, it feels that you too could easily become unjustly retained in a Big Brother state based on undefined risk data you have absolutely no idea about, and no recourse to challenge. It also deepens fears around relying on AI to make emotionally laden judgments and the extremes to which society might take surveillance tech. You can practically hear the clanging of emergency warning bells as you read.
A creepingly haunting warning of where the next evolution of technology could lead us.
Thanks to Knopf, Pantheon, Vintage and Anchor, and NetGalley for an advanced reader’s copy.