Nicholas Carr's forthcoming The Glass Cage, about the ethical dangers of automation, inspired me to read George Orwell's The Road to Wigan Pier (1937), which contains a lengthy tirade against the notion of progress as efficiency and convenience. Orwell declares that "the tendency of mechanical progress is to make life safe and soft." It assumes that a human being is "a kind of walking stomach" that is interested only in passive pleasure rather than work: "whichever way you turn there will be some machine cutting you off from the chance of working — that is, of living." Convenience is social control, and work, for Orwell at least, is the struggle to experience a singular life. But the human addiction to machine-driven innovation and automation, he predicts, fueled apparently by a fiendish inertia that demands progress for progress's sake, will inevitably lead to total disempowerment and dematerialization:
There is really no reason why a human being should do more than eat, drink, sleep, breathe, and procreate; everything else could be done for him by machinery. Therefore the logical end of mechanical progress is to reduce the human being to something resembling a brain in a bottle.
Basically, he sees the Singularity coming and he despises it as a "frightful subhuman depth of softness and helplessness." And there is no opting-out:
In a healthy world there would be no demand for tinned foods, aspirins, gramophones, gaspipe chairs, machine guns, daily newspapers, telephones, motor-cars, etc., etc.; and on the other hand there would be a constant demand for the things the machine cannot produce. But meanwhile the machine is here, and its corrupting effects are almost irresistible. One inveighs against it, but one goes on using it.
This "brain in the bottle" vision of our automated future, Orwell surmises, is why people of the 1930s were wary of socialism, which he regards as being intimately connected ideologically with the theme of inevitable progress. That connection has of course been severed; socialism tends to be linked with nostalgia and tech's "thought leaders" tend to champion libertarianism and cut-throat competitive practices abetted by technologically induced asymmetries, all in the name of "innovation" and "disruption."
Oddly, Orwell argues that the profit motive is an impediment to technological development:
Given a mechanical civilization the process of invention and improvement will always continue, but the tendency of capitalism is to slow it down, because under capitalism any invention which does not promise fairly immediate profits is neglected; some, indeed, which threaten to reduce profits are suppressed almost as ruthlessly as the flexible glass mentioned by Petronius ... Establish Socialism—remove the profit principle—and the inventor will have a free hand. The mechanization of the world, already rapid enough, would be or at any rate could be enormously accelerated.
Orwell seems to imagine a world with a fixed amount of needs, which technology will allow to be fulfilled at the expense of less labor; he imagines technology will make useful things more durable rather than making the utility we seek more ephemeral. But technology, as directed by the profit motive, makes obsolescence into a form of innovation; it generates new wants and structures disposability as convenience rather than waste. Why maintain and repair something when you can throw it away and shop for a replacement — especially when shopping is accepted to be a fun leisure activity?
While Orwell is somewhat extreme in his romanticizing of hard work — he sounds downright reactionary in his contempt for "laziness," and can't conceive of something as banal as shopping as a rewarding, self-defining effort for anyone — people today seem anything but wary about technological convenience, even though it is always paired with intensified surveillance. (The bathetic coverage of Apple's marketing events seems to reflect an almost desperate enthusiasm for whatever "magical" new efficiencies the company will offer.) Socialism would be far more popular if people really thought it was about making life easier.
Orwell associated automation with socialism's utopian dreams, and thought the flabbiness of those dreams would drive people to fascism. Looking back, it seems more plausible to argue that automation has become a kind of gilded fascism that justifies itself and its barbarities with the efficiencies machines enable. Though we sometimes still complain about machines deskilling us, we have nonetheless embraced once unimaginable forms of automation, permitting it to be extended into how we form a conception of ourselves, how we come to want anything at all.
One might make this case for automation's insidious infiltration into our lives: First, technology deskilled work, making us machine monitors rather than craft workers; then it deskilled consumption, prompting us to prefer "tinned food" to some presumably more organic alternative. Now, with the tools of data collection and algorithmic processing, it deskills self-reflection and the formation of desire. We get preemptive personalization, as when sites like Facebook and Google customize your results without your input. “Personalization” gets stretched to the point where it leaves out the will of the actual person involved. How convenient! So glad that designers and engineers are making it easier for me to want things without having to make the effort of actually thinking to want them. Desire is hard.
Preemptive personalization is seductive only because of the pressure we experience to make our identities unique — to win the game of having a self by being “more original” than other people. That goal stems in part from the social media battlefield, which itself reflects a neoliberal emphasis on entrepreneurializing the self, regarding oneself as leading an enterprise, not living a life. If "becoming yourself" was ever a countercultural goal, it isn't anymore. (That's why Gap can build an ad campaign around the proposition "Dress Normal." Trying to be distinctive has lost its distinction.) It's mandatory that we have a robust self to express, that we create value by innovating on that front. Otherwise we run the risk of becoming economic leftovers.
Yet becoming "more unique" is an impossible, nonsensical goal for self-actualization: self-knowledge probably involves coming to terms with how generic our wants and needs and thoughts are, and how dependent they are on the social groups within which we come to know ourselves, as opposed to some procedure of uncovering their pure idiosyncrasy. The idea that self-becoming or self-knowledge is something we’d want to make more “convenient” seems counterproductive. The effort to be a self is its own end. That is what Orwell seemed to think: "The tendency of mechanical progress, then, is to frustrate the human need for effort and creation."
But since Orwell's time, the mechanization process has increasingly become a mediatization/digitization process that can be rationalized as an expansion of humans' ability to create and express themselves. Technological development has emphasized customization and personalization, allowing us to use consumer goods as language above and beyond their mere functionality. (I'll take my iWatch in matte gray please!) Social media are the farthest iteration of this, a personalized infosphere in which our interaction shapes the reality we see and our voice can directly reach potentially vast audiences.
But this seeming expansion of our capacity to express ourselves in in the service of data-capture and surveillance; we embed ourselves in communication platforms that allow our expression to be used to curtail our horizons. Preemptive personalization operates under the presumption we are eager to express ourselves only so that we may be done with the trouble of it once and for all, once what we would or should say can be automated and we can simply reap the social benefits of our automatic speech.
Social media trap us in a tautological loop, in which we express ourselves to be ourselves to express ourselves, trying to claim better attention shares from the people we are ostensibly “connecting” with. Once we are trying to “win” the game of selfhood on the scoreboard of attention, any pretense of expressing an “inner truth” (which probably doesn’t exist anyway) about ourselves becomes lost in the rush to churn out and absorb content. It doesn't matter what we say, or if we came up with it, when all that matters is the level of response. In this system, we don't express our true self in search of attention and confirmation; instead attention posits the true self as a node in a dynamic network, and the more connections that run through it, the more complete and "expressed" that self is.
When we start measure the self, concretely, in quantified attention and the density of network connectivity rather than in terms of the nebulous concept of “effort,” it begins to make sense to accept algorithmic personalization, which reports the self to us as something we can consume. The algorithm takes the data and spits out a statistically unique self for us, that lets us consume our uniqueness as as a kind of one-of-a-kind delicacy. It masks from us the way our direct relations with other people shape who are, preserving the fantasy we are sui generis. It protects us not only from the work of being somebody — all that tiring self-generated desire — but more insidiously from the emotion work of acknowledging and respecting the ways our actions have consequences for other people at very fundamental levels of their being. Automated selfhood frees us from recognizing and coping with our interdependency, outsourcing it to an algorithm.
The point of “being unique” has broadened; it is a consumer pleasure as well as a pseudo-accomplishment of self-actualization. So all at once, “uniqueness” (1) motivates content production for social-media platforms, (2) excuses intensified surveillance, and (3) allows filter bubbles to be imposed as a kind of flattery (which ultimately isolates us and prevents self-knowledge, or knowledge of our social relations). Uniqueness is as much a mechanism of control as an apparent expression of our distinctiveness. No wonder it's been automated.