Monday, July 25, 2011

Condemned to be Free?

In my last post I discuss weak links, and explore the idea that as society and capitalism develop, those links will disappear, so everything will coalesce into a single, strongly linked network. We will all share the same information and culture, and consume the same products. But there is an alternative possibility, which is that being awash in so much information, we each will filter it in our own way. We all draw from the same bottomless pit, but each of us seeing different cuts of that information, and thus, post-filtering, occupying different worlds.
A recent book, The Filter Bubble, discusses the mechanics and the implications of such personalized filters. It views an emerging world where there are filters between us and the Internet that are developed by entities beyond our control, tapping a database of our searches, purchases, and “friends”. Emerging might be too tentative a word. Personalized filtering already exists. Witness the fact that your Google search will bring up different results than will mine; that the friends highlighted on your Facebook account, or the e-mails that percolate up to be deemed important, or the stories that arrive in your news feed, not to mention the obvious, the ads that you see, are already based on these filters. The filters color our view of the world and serve us the food for our thoughts.
There is a good reason for filters. We are hopelessly engulfed by an ocean of information. Eric Schmidt has said, "There was 5 exabytes [an exabyte = a quintillion bytes] of information created between the dawn of civilization through 2003, but that much information is now created every 2 days, and the pace is increasing”. (Though not everyone agrees with this, and in any case, a lot of it is made up of inane YouTube videos and forgettable tweets, e-mails, friend updates, …and rehashed blog posts).
Personalized filtering started off to better target advertising, though I wonder if a longer term result will be for personalized filtering to destroy the advertising-based business model widely employed in the search and social networking arena. Businesses pay to advertise on sites like Google and Facebook in order to get out information about their products. Yet the raison d'etre for the likes of Google and Facebook is as a platform for finding and distributing information. If the information that is being distributed in the course of their business is the same information that the advertisers are paying them to distribute, then these sites are feeding on their own seed stock. And this circumstance is made more likely because of the increasingly widespread use of personalized filters.
What happens if the personalized filters get us what we want to know without anyone having to pay to tell us? Will the free flow of information dominate the value, (it certainly will dominate the amount), of the information the advertisers pay to distribute?
I bet that once these algorithms are in full bloom, you will be able to type in “I need a new pair of sandals for the beach. What should I get", or “what are some fun things for me to look at in terms of accessories”. (Or maybe even “I'm bored, have two hours to kill, so what should I do right now"). The algorithm will look at what you have bought in the past, and what of that you have returned versus reordered, bought on sale or were willing to buy at full price. It will look at the same for your friends, weighted by their closeness to you and their relevance for your purchase. If you spend time keeping up with Shakira, it will look at what she and her circle are up to. It will look at a lot more as well, and then up will pop a bunch of links tailored for your preferences. As the algorithmic searches get better and better, the paid ads will be increasingly irrelevant.
I want to go beyond the prosaic issue of the implications of personalized filtering for the future of advertising to think about a deeper, existential issue: how personalized filters might affect our notion of individual freedom.
Memory of All Things Past
Ireneo was nineteen years old; he had been born in 1868; he seemed as monumental as bronze, more ancient than Egypt, anterior to the prophecies and the pyramids. It occurred to me that each one of my works (each one of my gestures) would live on in his implacable memory; I was benumbed by the fear of multiplying superfluous gestures. – Jorge Luis Borges' “Funes, the Memorious”, translated by Anthony Kerrigan.
For better or worse, there is a lot of information about you – events, associations, purchases and travel, and a lot of communications and pictures – that is stored somewhere or other. If all goes according to plan, we will not only know everything that is going on in the world day by day, we will have a perfect memory of the past. Surprisingly, the result, a virtual version of perfect memory, can be a problem, and the only way out of that problem is to employ a filter of some sort.
In Borges' Funes the Memorious, Ireneo Funes recovers consciousness after a fall from a horse to discover that he has perfect memory. He can recount ever facet of every day, he can summon a vision of every person he has met from every perspective in which he saw them. He finds it difficult to come to terms with the fact that those different people, each recalled in full detail and within all the events surrounding the encounter, are in fact the same person viewed at different times.
Borges wrote that Ireneo was “almost incapable of general, platonic ideas. It was not only difficult for him to understand that the generic term dog embraced so many unlike specimens of differing sizes and different forms; he was disturbed by the fact that a dog at three-fourteen (seen in profile) should have the same name as the dog at three-fifteen (seen from the front). His own face in the mirror, his own hands, surprised him on every occasion. “
Funes's world was a jumble of the particulars that affected his ability to think, because thinking requires seizing on what is important to the context and, at least for the moment forgetting the rest. (Related to this, there was a medical study that demonstrated an inability to filter out memories causes decreased mental performance and lower intelligence, though I can't remember the details).
For a real-life example there is the mnemonist Solomon Shereshevsky, who simply went by “S”. S. was a newspaper reporter. One day his editor gave him lengthy instructions of places to go, people to meet and information to gather. The editor, noticing that S. had not written down any of the instructions, was about to reprimand him for his inattentiveness, but S. repeated back everything exactly that he was assigned to do. The editor probed him more closely on his memory, and amazed by his abilities, sent him to a psychological laboratory that was focused on the investigation of memory. There S. met the psychologist Alexsander Luria, who would study him for the next thirty years and write about the process behind his mnemonic gifts.
Luria found himself unable to measure the capacity of S.'s memory. S. could surmount anything Luria put in his path. Luria read to him as many as 30 words, numbers, or letters and asked him to repeat these. He recalled them all. Luria increased the number to 50, then to 70, but S. still could recall them and even repeat them in reverse order. It didn't matter whether he was presented words with meaning or nonsense words. Fifteen years after their first meeting, Luria asked S. to reproduce the series of words, numbers, or letters from that first meeting. S. sat, his eyes closed, and recalled the situation:
“yes-yes … this happened with us at that apartment … you sat at the table, and I was in the rocking-chair … you were wearing a gray dress suit and looked at me like this … here … I see what you were saying to me …” – and afterward followed an flawless reproduction of the read row. Even more amazing is that by that time S. had become a famous mnemonist and was required to remember many hundreds and thousands of rows.
As with Ireneo, S. had a poor memory for faces. "People's faces are constantly changing," he said; "it's the different shades of expression that confuse me and make it so hard to remember faces" Details that other people would forget occupied his mind, making it hard to move from the flow of images and sensations to some higher level of awareness such as abstraction, and meaning. He perceived the changes of faces the way we might perceive constantly changing light and shadow, like looking at the ripples of a pond or eddies of a river.
Also, like Ireneo, with a memory composed entirely of details, S. couldn't think on an abstract level. He could recite a story word for word, but could not easily summarize it. When required to go beyond the information given, such as understanding metaphors, puns or symbolism, S. was out to sea.
If a story was read quickly, S. would become perplexed, “No this is too much… every word gives rise to images and they find each other and there is chaos…I cannot make anything out of it..and there is also your voice and also spots…and everything blends together...”
Escape from Freedom
I suspect...that he was not very capable of thought. To think is to forget differences, to generalize, to abstract. In the overly replete world of Funes there were nothing but details, almost contiguous details. – Jorge Luis Borges' “Funes, the Memorious”, translated by Anthony Kerrigan.
Perfect memory impairs the mind's ability to abstract, to infer, and to learn. Moreover, the nature of memory is not simply storing and retrieving. It is also part of the creative process. We make up memories through inferences and reconstruct them into new ideas.
Yet we are developing perfect memory. Perfect memory of everything that has happened anywhere, that anyone has experienced. For a computer, nothing is lost. We could end up like S., knowing everything and as a result lose all context. So the next step for the Googles of the world (which is pretty much to say, Google) is to find a way to distil and synthesize that information.
Cognitive limitations both constrain and enable adaptive behavior. There is a point where more information and more cognitive processing can actually do harm, the possibility of perfect memory giving us the extreme case. Built-in limitations can in fact be beneficial, enabling new functions that would be absent without them.
Imagine sitting by the fireplace with Ireneo as he recounts an amusing incident from his past. He relates every single fact related to the event. Every detail of the room, every gesture of all those attending, each word spoken. Without the filtering, the story is lost, or, more precisely, it is stripped of context. There is no way to tell, in the description he provides, what really matters. Such a story is meaningless, as is such a life. We need filters to think and to experience the world.
Perfect memory is like having all the past also be the present which in turn is like living totally in the trivial of the present. And not just learn something new, but also view the object with a different perspective.
To see an object in a thoughtful way we ignore some of its characteristics; we let some things come into focus while letting the rest recede into the background. We encounter and filter things based on our objectives and interests; the tree of a botanist is not the tree of the artist. So, more broadly, the world is meaningful to us only when we see it in a context, and this means electing to ignore some aspects of it. Meaning emerges only by highlighting some features while relegating to the shadows those that are irrelevant to our context.
The world we create through our willing and selective ignorance is what Sartre calls “nihilation”. We are like a sculptor that creates his work by removing part of a slab of stone. To see a world of individual things meaningfully related to one another is to elevate part of the perceptual field to the foreground and to relegate other parts of it to an undifferentiated background. Otherwise it all exists too completely; some parts of that existence have to be negated for it to become the subject of thought. To know everything, to remember everything, is not the mark of intelligence.
To think, we first choose what to look at and what to ignore. This is what Sartre meant by the statement “we are condemned to be free”. In taking over this task, the personalized filter takes our freedom as well. If filtering is part of thinking, then taking over the filtering also takes over how we think.
Conclusion
Filtering is essential to cognitive performance and intelligence. It determines our behavior and is an expression of our free will. Up until recently, our brains were the only filters, and then we started to use machine-based filters (to do things like pick the characteristics for people who would pop up on a dating site). Those filters were were simple and easy to understand. But now these artificial filters are becoming so complex and pervasive that they are performing our precognition, at least as far as what we see and do online.
There may be some merit to computers taking over this function. Maybe they can do it better. Perhaps these filters are just computationally more powerful versions of our own cognitive filters, just as computers already are more powerful versions of other cognitive functions. But the filters are not under our control, we may not even be aware that they are whirring around, and even if we do, we won't understand how they are deciding what to allow through the filter for our consideration. Of course, it is not as if we are about to be subjugated by self-aware computers. Not self-aware. But aware of us, intimately so.

Tuesday, July 19, 2011

LinkedIn Weak Links

I know a guy...”
Is LinkedIn anything more than a resume warehouse? Is it simply a web-based Rolodex to make it easier to stay up with your business contacts?

I don't know the business model for the company, and don't have any basis to comment on its long-term prospects, but in a somewhat stream-of-conciousness mode, I hear the “linked” in LinkedIn, and it starts me thinking about network theory, and the place LinkedIn most naturally fills in that space, which is to establish weak links.

In partiular, LinkedIn is an application of a theory the sociologist Mark Granovetter proposed in a 1973 paper. Granovetter looked at how job seekers used their social connections in their search. He came up with the surprising finding that the most valuable networking contacts in a job search are not from the ranks of close relationships, but rather from casual acquaintances and those known only indirectly through others.

The reason these second-order relationships, which he calls weak ties and which network theory refers to as weak links, are so valuable in the job search is that they have information that is outside the sphere of the tighter circle of friends. That is, when it comes to job opportunities, your close associates are likely to know pretty much the same things that you know already, and for that matter, any one of them is likely to know most of what any other one knows. After all, you are close because you work in similar jobs, went to the same schools, live in the same neighborhood. But the weak links extend your reach to opportunities that you and your strong links do not know.

If LinkedIn is successful in helping to create and exploit weak links, it also can be effective in fostering group action. Just as a job search depends more on weak links than on one's immediate group, so political action depends on weak links to spread the word and enlist broad action. This is true whether the means of communication is word of mouth, leaflets or the Internet. Granovetter discovered this when he examined the Italian community of Boston's West End to try to explain their lack of success in creating effective community organizations. He attributed it to strongly linked clusters or cliques that lacked weak links connecting them. The result was that any leader seeking to form a coalition to oppose, say, an urban renewal project, found no way into the various cliques. Starting with any one group, there might be no one who knew someone in another group well enough to provide an introduction, or because each group was so tightly linked and suspicious of outsiders (which would explain the lack of weak links) the leader could not generate the requisite trust.

Weak links are a key source of stability and resilience in networks. They are the conduit for sources of information and support while not being essential to the network's function. By definition, a weak link is one that is not integrated into a network; it can be broken without affecting the function of the network Their value derives in part from being unessential, and, for that matter, low maintenance. Having sets of strongly linked networks that are then linked to one another through weak links is an attractive structure for survivability.

One question that is widely debated is if LinkedIn, social network sites in general, and more broadly the expansion of information and communication, bind us more strongly. Do they affect the number and the value of weak links?

My view is that there are forces that govern the number of weak links, and the structure of networks generally, that occur even in their absence. In particular, I propose the following hypothesis:
Over the course of social evolution, the number of weak links initially increases and then, as the society matures, decreases. The same is true over the course of capitalist evolution.

To see what I mean by this hypothesis, which I will expand on below, consider the progression from the starting point of society. Early society consisted of small networks of closely connected nodes – the family, the village. These groups then begin to connect to one another through weak links. In some cases, the node – the person – who provides that connection becomes as important as the central figure in the group. But if the weak links have persistent value, they will strengthen and the network will expand to include that link and its related network. The end result is a society and economy that is integrated, and that, because of the loss of weak links, is less diverse and resilient. Constraining how far this goes is as much technological as it is culture: How many balls can anyone have in the air? How well can they communicate? Our Facebook's thousands of friends and LinkedIn's hundreds of connections notwithstanding, the fact is we really can't be friends and colleagues with all the matter in the universe.

Networks, Weak Links, and the Evolution of Society
Anthropologist Allen Johnson separates the development of society into three stages: the family/tribe, the local group, and the regional polity. Johnson views these three through a set of eight variables. The quality of network linkages is not one of them. But my objective is to provide a perspective for the role of the likes of LinkedIn, so I will take his three stages along a different route.

In the first stage, the family/tribe, all of the members are tightly linked. There are few if any weak links to other groups or tribes. The unit size is on the order of fifty people There are abundant resources for the tribe to forage or to develop the beginnings of agriculture and herding, with population density that is less than one person per square mile. With density so low, there is little need for territorial defense.
As population grows and resources become more constrained, the family/tribes form local groups in response to population pressures and the need to shepherd and protect resources. The network structure is a cluster of various tightly linked family-tribes all linking into a central figure, termed a Big Man.

Local groups might have hundreds of people and a population density that is ten times that of the family/tribe. This is near the limits for the number of strong links one person can manage, (embodied in the concept of the Dunbar number), and so weak links may start to emerge within the Local Group. Protection of those resources requires keeping other tribes at bay, so interaction with others, when it does occur, is often violent rather than cooperative. But it is within the local groups that weak links begin to form as they reach out in informal trade, and begin the transfer of technology, such as methods of farming and storage, toolmaking and the like. The result is greater stability than for the isolated family/tribe.

The development of weak links continues as the Local Groups evolve into Regional Polity, which includes the chiefdom and the state. For the Regional Polity, the value of weak links across chiefdom becomes manifestly evident, because of the increased importance of trade and alliances. The nodes that create the weak links with other Regional Polities define the nature of the state, for example differentiating feudal, capitalist and totalitarian states.

For the feudal state, the links come through emissaries of the state and the bonds of marriage. That is, the weak links are maintained under the control of the central node of the network. If the weak links across the chiefdom and states come through merchants and financiers, we have nascent capitalism. The importance of the weak links can be measured by observing the stature and power afforded to the merchant class, which controls those connections. In the totalitarian state, the ideal network is to have all members connected through a strong link with the center and have weak links across the members themselves. Of course, it is impossible for the center to establish and maintain the strong links directly, so they do so through the political machine or covertly through the state police, both of which also take action to keep any intra-member links weak, or prevent their establishment at all.

Networks, Weak Links, and the Evolution of the Political Economy
The evolution of society is carried forward hand in hand with the evolution and stratification of the political economy. For example, until around 1100 AD Europe was peppered with small towns and villages, each self sufficient and nameless. Transportation was perilous, and was taken with a risk of becoming forever lost. And, in any case, there was little reason to travel from one village to the next, given that one had little to offer compared to another. (Sort of like each town having its own McDonalds, Starbucks, and Gap).
The exception was the towns that were in a central position, perhaps based on being on a waterway or adjacent to a geographical pass, and thus became the hub for surrounding villages. Over time these towns grew in wealth, and the wealth in turn supported guilds, which then over time became specialized based on their proximity to the raw materials for metals, wood, and textiles. The guilds provided goods to enhance the lives of those in the adjacent country side, and they in turn provided the towns with food and raw materials. Thus developed the economic structure for the Local Group.
The town and its supporting countryside each represented its own network of associations, and it became the task of the merchants to provide the links between them. The reason such links made sense post-1100 where they didn't a few centuries earlier was the differentiation of production that supported trade. The links were weak because, first, it was only through the merchants that the connection existed, and second, if the merchants disappeared, the basic function of the town continued as before.
As towns with proximity to waterways became port cities and hubs of trade, the next step of capitalism developed. The product of the port cities was their links rather than real goods. And while the interior merchants plied their wares, trading the goods of one town for those of the next, it is in the ports that the essentials of capitalism took form: larger scale trade required capital for ships and letters of credit to finance cargo.
The point is that the evolution of society was often led by the network and links created by the economy, or was stifled by the lack of such links. The deliberate isolation in the Tokugawa era of Japan met its end because of the links that came about through trade. The control of wealth by the feudal lords began to fade, much as it did in the West, as the capital locked in the land was unleashed for capitalist enterprise, and the land-centered production was overshadowed first by that of the merchant class, and then of the industrialist. Another extreme is the early Ming in China, which demonstrated legendary seafaring, with fleets of 27,000 men traveling during multi-year expeditions to India and beyond, in one case as far as Africa. But the interest was amassing tribute and curiosities, (including giraffes!), not trade, with the result being centuries of isolation which left them in the dust during a period of unprecedented development in the West.

I'd Like to Teach the World to Sing
A key difference between a world made up of democratic, capitalistic societies and a world made up of feudal, totalitarian societies is the evolution forward from the Regional Polity. Absent revolution or war, the feudal state, centered around a ruler who guards all weak links, will be stable. Other states will be kept at arms length, with the only link being a weak link from ruler to ruler expressed through treaties and trade agreements. If, on the other hand, the links are established between the merchants and bankers, the links between the states may move beyond the ruler’s control, and will become increasingly strong. More information flow is better for business, as is a broader network for channels of production and distribution of product. Ultimately, at least in economic terms, the boundaries of the Regional Polity blur, and rather than having network clusters connected via weak links, the clusters integrate into one tightly connected network.

The key to this process is the limits of technology. Until very recently, the cost of communication and the limitations on keeping track of contacts was the constraint on transforming weak links into strong ones. But now the limitations of communication and processing are fading. The result is one large state, not in political terms, but in economic and informational terms. LinkedIn is one instrument in this process, but it extends beyond the social networks. Indeed, as I have argued in a recent post, the impact of links arising through much of the social media is more apparent than real. However, it is the most visible manifestation of what is occurring.

As the societal structure throughout the globe has moved along this path, the number of independent units has fallen. During Neolithic times there may have been in excess of 100,000 Local Groups, while today in we have a Regional Polity stage with only a few hundred states. And if we think in economic rather than political terms, the clusters, though difficult to define, are no doubt far smaller, and may be shrinking still.

From a network perspective, we are then all one big, happy, strongly linked family and that is the process of pulling in all information, to the point that there is less and less for a weak link to give you. In terms of information, everyone will be strongly linked, in the sense that they will all have the same (social) information.

So do we at last all link hands (strongly) and sing in perfect harmony? Even absent the political differences that prevent integration on the informational and economic side from extending out, there is a problem with a network that moves from many small networks connected by weak links to one network largely connected through strong links. And that is a reduction of resilience. Recall that a weak link is, by definition, one which at once both provides information from a new source and is not disruptive if it is broken. Recall also that in the earliest stage of society, the family or tribe, the key vulnerability was its isolation and lack of diversity. The final stage in the evolution of society brings things full circle, because although much larger and more productive, the fully linked society, with everyone sharing the same network, is also less resilient. All the nodes and clusters will tend to come to the same world view, share and depend on the same technology, and become intertwined in production. I wrote about this tendency in an earlier post on “asexual capitalism”.

And this perhaps gives rise to the collapse of societies. There is no end to speculation and proposals for the reason societies, flourishing and with great power and resources, falter and collapse. One suggestion, by Joseph Tainter, is that they fall under their own weight, they require ever increasing amounts of energy – both for operation and maintenance, and for management oversight – until the populace realizes it might be more efficient and productive to break away. With that realization usually precipitated by shocks such as famine or invasion. But another might be the reduction in weak links as the civilization expands, enveloping and integrating its neighbors, links that might have introduced the civilization to other ideas and approaches, warned of impending disaster, given shelter during calamity, and provided the resilience to these shocks.

The Securities and Exchange Commission disclaims responsibility for any private publication or statement of any SEC employee or Commissioner. This post expresses the author's views and does not necessarily reflect those of the Commission, the Commissioners, or other members of the staff. Similarly, this post expresses the author's views and does not necessarily reflect those of the Department of Treasury or its staff.

Saturday, July 16, 2011

Fifteen Years of Brazilian Jiu Jitsu

If you read to the last chapter of my book, A Demon of Our Own Design, then you know that I like Brazilian Jiu Jitsu (BJJ). I have just passed the fifteen year mark of studying at the Renzo Gracie Academy in New York. During that time I have seen the academy grow from twenty or so students, most all beginners, to a thriving center with hundreds of students and a dozen world-class instructors, most of whom started their careers with Renzo and began studying jiu jitsu well after I did. I have had the opportunity to study with several of them (linked here to their respective academies) – Shawn Williams, Brian Glick, and the inimitable John Danaher – though through the combined limits of age, talent and time, my development has followed what might charitably be called a slow trajectory.

Here I am with Brian Glick (to my right) and John Danaher. (In case your wondering, I photoshopped out some stuff on the shirts).


Being over sixty, I am an anomaly; most of those who train are in their twenties and thirties. But I don't see why that should be the case. From a physical standpoint, BJJ is low impact. There are no sudden and explosive moves, there is time to adjust or, if things are not going right, to “tap out” and start over from a neutral position. Over the course of fifteen years, my worst injury has been a pulled hamstring. Though, granted, I have been careful to train with thoughtful and technical partners, something that I would encourage anyone to do who is past their physical prime.

BJJ also does not require speed or strength, (though of course both help, as does flexibility and muscle memory), and it provides what an “as seen on TV” testimonial would describe as a “full body workout”. BJJ particularly develops core strength and flexibility, and also develops body awareness. You have to sense what is where when you are intertwined like a pretzel or are rolling away from a move, as well as become aware of what parts of the body have to be engaged in any given situation and relax those parts of the body that do not in order to stave off exhaustion.

I have another theory about the physical benefits of BJJ. BJJ comes as close to mortal combat as just about anything we can do in modern society without having injury as inevitable. In live training, there is no holding back; none is necessary because the techniques have been developed with the objective of allowing such training. But the end point is a submission, typically by way of a choke hold or a joint lock. As I mentioned, you tap before sustaining an injury, but my hypothesis is that at a deeper level your body senses it is in danger and the result is a physiological response which maintains the body in a heightened state of physical capability.

Which gets to another value of BJJ. As anyone who has watched mixed martial arts knows, BJJ is an integral part of the competition. In the early days, before it became a part of everyone's arsenal, it dominated other approaches. Being on the ground neutralizes many components that would give the advantage to the stronger and faster opponent, and thus it maximizes the advantage of the one with the greater technical knowledge. By comparison, in standup fighting the edge gained through training and technique can only go so far. Witness the brawlers who still can work their way to the upper ranks in professional boxing.

Some people might roll their eyes when they hear the term “martial art”. But there is art – and science and intellectual play – in BJJ, though as with many other art forms, it may not be apparent to the untrained eye. BJJ is an art in the same way as is chess; the more one studies and becomes immersed, the more there is to appreciate in the moves and strategy. Because of this, BJJ can be engaging as a life-long pursuit.