“I don’t understand how a computer can not function.”
That was a comment by Patricia Minchin, a 75 year old nurse who was one of the tragically unfortunate people who was missed out of breast cancer screening and now has untreated advanced breast cancer. Yet another NHS fiasco. The Secretary of State has blamed it on a malfunctioning algorithm.
With all the talk about the relentless march of AI, How computers are going to take over that part of the world that they have not already taken over, destroy jobs and give us all sorts of promised benefits, we seem to have lost sight of one main issue – computers and the algorithms that drive them are prone to error.
And the consequences of those errors can be devastating as in the case of all the women who die unnecessarily because of a ‘malfunctioning algorithm’, those who will lose their lives when self-driving cars don’t work properly, and endless other examples we can come up with.
Years ago I remember watching a cute film called Sleepless in Seattle. Two kids were hatching a scam to get one of them to fly to New York unaccompanied. To do that, they had to falsify the age of the traveller. “They’ll never believe I’m twelve years old,” quips one of them. “If it comes out of the computer, they’ll believe anything,” responded the other.
This is the fundamental problem that we are all facing. The hype around technology and what it can do for us has turned us into a flock of sheep being led by the nose by anything that comes out of a machine driven by a supposedly intelligent algorithm. The fiascos – be they the NHS, the TSB, Facebook with its fake news and uncontrollable platform – don’t seem to have shaken our faith in a future technological nirvana.
We seem to have forgotten that algorithms are written by fallible humans. That when any platform becomes a jumble of algorithms written by different people art different times, the output may end up being a total mess.
There is little doubt that automation and technological process can bring great benefits. But we should also bear the dangers in mind. We should not all be quite so quick in believing that anything that comes out of a computer is necessarily clever or right. It’s not clear how we are going to be able to balance the benefits of technology with maintaining some degree of transparency as to what is actually going on inside our machines. What assumptions and value judgements have been incorporated into what seems like an innocently clear output. What unforeseen interactions are going on in ever more complex and opaque systems driven by multiple algorithms.
This will be a challenging issue. But we need to face it and talk about it a lot more that we are if we are not to move towards a world where machines instruct us to turn right or left and we have absolutely no idea where that instruction came from, and whether it is what we really want to do.
Help us lay the intellectual foundations for a new radical politics. Sign up to get email notifications about anything new in this blog.
Joe, indeed, and a similar message to that we’ve been discussing with our work, and indeed recent survey, on how people feel about a Digital Society (https://itunes.apple.com/us/podcast/human-potential-at-work/id1110960287?mt=2, #105). On the back of that we’ve begun to focus in on #CorporateDigitalResponsibility, and to consider how organisations should align to deliver improved outcomes for society through the use of Digital technologies (inc. AI and more)