"But in reality, there is no reason to assume that artificial intelligence will gain consciousness, because intelligence and consciousness are very different things. Intelligence is the ability to solve problems. Consciousness is the ability to feel things such as pain, joy, love and anger. We tend to confuse the two because in humans and other mammals intelligence goes hand in hand with consciousness. Mammals solve most problems by feeling things. Computers, however, solve problems in a very different way."

“Yet from the viewpoint of the herd, rather than that of the shepherd, it’s hard to avoid the impression that for the vast majority of domesticated animals, the Agricultural Revolution was a terrible catastrophe. Their evolutionary ‘success’ is meaningless. A rare wild rhinoceros on the brink of extinction is probably more satisfied than a calf who spends its short life inside a tiny box, fattened to produce juicy steaks. The contented rhinoceros is no less content for being among the last of its kind. The numerical success of the calf’s species is little consolation for the suffering the individual endures.”

“I cannot name the 8 million people who share my Israeli citizenship, I have never met most of them, and I am very unlikely ever to meet them in the future. My ability to nevertheless feel loyal to this nebulous mass is not a legacy from my hunter-gatherer ancestors, but a miracle of recent history.”

“At first sight, domesticated animals may seem much better off than their wild cousins and ancestors. Wild buffaloes spend their days searching for food, water and shelter, and are constantly threatened by lions, parasites, floods and droughts. Domesticated cattle, by contrast, enjoy care and protection from humans. People provide cows and calves with food, water and shelter, they treat their diseases, and protect them from predators and natural disasters. True, most cows and calves sooner or later find themselves in the slaughterhouse. Yet does that make their fate any worse than that of wild buffaloes? Is it better to be devoured by a lion than slaughtered by a man? Are crocodile teeth kinder than steel blades?”

“As algorithms come to know us so well, authoritarian governments could gain absolute control over their citizens, even more so than in Nazi Germany, and resistance to such regimes might be utterly impossible. Not only will the regime know exactly how you feel – it could make you feel whatever it wants. The dictator might not be able to provide citizens with healthcare or equality, but he could make them love him and hate his opponents. Democracy in its present form cannot survive the merger of biotech and infotech. Either democracy will successfully reinvent itself in a radically new form, or humans will come to live in ‘digital dictatorships’.”

“If we are willing to make such efforts in order to understand foreign cultures, unknown species and distant planets, it might be worth working just as hard in order to understand our own minds. And we had better understand our minds before the algorithms make our minds up for us.”

“: 4: Increasing the freedom of movement along existing connections.”