Tuesday, November 25, 2014

Uber: What could possibly go wrong

Uber at the core is nothing more than a bunch of guys who came up with a clever app for streamlining the way you call for a car.  Rather than phoning or texting, you tap on the app.  Rather than paying with cash or credit cards, you have an account.  And rather than depending on the reputation of the car company, you depend on the ratings-based reputation of the specific driver. (Another feature, the pre-determined fare, with that fare essentially including all fees including tip, is nothing new for car services). And, by the way, it is readily replicable by any other car services, including those that already have a reputation and experienced drivers.  

So it seems to me that it is only mildly innovative.  How do you build a successful world-wide  company based on this? I don’t think you do. We are already seeing some of what can go wrong.  We can attribute a lot of this to their own doing. But we are only getting started. Here are four other things that come to my mind. 

A meta app. If a number of Uber competitors, existing limo companies, and taxis put out similar apps, (which they are), the next stage will be a meta-app that looks across all of the companies, scans their car locations and prices, and combines this with the user’s preferences to find the best deal. I would bet that Uber will decline to be involved, but if a broad base of other limo companies do, and it ends up being Uber versus the world, I don't think that will turn out well for Uber. 

Uber’s costs are underestimated. To some degree Uber is thriving by creating a regulatory arbitrage.  It is operating without the constraints of existing car services and taxis. That gives it lower costs. But that is not going to last. The arbitrage window will close, and the related cost advantages will disappear.

The first out of the gate usually doesn’t win.  Uber is doing all the heavy lifting in terms of clearing regulatory barriers and local opposition, and getting consumers warmed up to the idea of using an app for calling cars.  It has made life harder for itself by defining its business as being more than an app and fighting its way through this morass.  But as many business school case studies have demonstrated, the company that blazes the trail is often not the one that stakes the biggest claim in the end.  If could be a set of future entrants.  Or it could simply be efficiencies that ultimately are picked up by existing car service without any one winner.  

Self-driving cars.  I’m not kidding -- and neither is Google.  Self-driving cars are, though it is hard for me to believe, predicted to become a consumer reality in five to ten years.  With these cars, the concept that is gaining momentum via Uber will become a reality.  Tap on your app, and a car will quickly appear to take you on your way.  This is not so far in the future, especially in the time scale of a company. The timeframe for profitability extends far into the future; indeed, many companies with high valuations don’t even turn a profit until five or ten years out. By then, the essence of Uber’s model might become irrelevant. 

People are attributing to Uber as a limo company what is really an app, and a new attitude by consumers towards car services that has come with it. If you have discovered a new and liberating method of obtaining transportation services, that is one thing, and you can thank Uber for bringing that to you.  But ultimately you won’t need an Uber driver for using it.  

Wednesday, November 19, 2014

On Death with Dignity

We take the will to live as a positive attribute, and naturally so; it is, of course in our genes.  Whatever subset of our species took a lackadaisical view toward the prospect of death was screened out long ago. We look with admiration on those who battle against terminal illness --  “She’s a real fighter” -- and look down on those who take their own lives. Fighting for your life is the genetic prime directive.   


Thus the recent death of Brittany Maynard spurred a flurry of news articles and posts on the Death with Dignity movement. There will be a lot more to come on this topic, and the movement will gather steam because whatever is in the crosshairs of the baby boomers weighs down on society as a whole. Soon the baby boomers will come to the last stage of their demographic wave, a wave that has progressed and altered society as it has moved through their life cycle, from crowding elementary schools into split session, to overcrowding colleges, to buying houses to retiring. The last area they can affect is the mode of dying, so this will become a topic we will increasingly hear about over the next few decades.  And ultimately some form of Death with Dignity will become standard in the US.  


I have weighed in on another medical issues in one of my past posts, making the controversial suggestion that people be compensated for some of the savings incurred if they select a cheaper but less efficacious course of treatment. Here I am going to barrel ahead with another one: a short vignette where I place the alternative to Death with Dignity in a disturbing but I hope illuminating context, that of a torturer and victim.  The victim is the one struggling with a painful, debilitating terminal illness.  The torturer is in part nature, in part the social norms that insist death proceed along its natural course, and in part the medical community that might be taking the Hippocratic Oath too literally. (And that, unfortunately, has a conflict of interest in keeping a dying patient in their revenue stream):


A particularly invidious development in techniques for torture, more common than many realize -- or are willing to admit -- starts with inducing periods of searing pain while a physician stands at the ready to assure the victim does not succumb to the torture and die prematurely, and indeed that he remains as alert as possible.  I mention dying prematurely because the spectre of death is another standard component of this brand of torture.  At the start, the victim is advised that no matter what transpires, the process will end in death.  Indeed, there is a program for increasing the stages of torture until death occurs. This program is determined before the torture has begun. The victim is given only a rough estimate of when termination will occur; the randomization is added so the victim remains in a state of uncertainty; this to add a psychological element to the torture.


The torture has been “improved” over time to match the physical pain with other aspects of psychological terror.  Probes are skillfully inserted into the victim’s brain, and slowly but steadily the victim’s motor skills are degraded.  Depending on the approach, this can occur by sequentially inducing paralysis -- first by subtle tremor, then seizures, and eventually ending in full paralysis, starting with the extremities, but then moving to the bowels and finally the lungs.  (This is a more clinically desirable replacement for the old school method of dismembering the extremities one at a time; the effect is functionally the same, but can be controlled to progress in a more psychologically devastating manner). More recently, thanks to better mapping of the brain, the degradation of motor skills has been accompanied by degradation of mental abilities, ranging from memory and speech to broader cognitive function. (Which, of course, cannot be allowed to progress too far too quickly, lest it inhibit the victim's awareness).


A common practice is to force the victim’s family members to observe the torture at every step, and even to encourage them to try to comfort the victim, but of course this is done with little effect as the torturer progresses the pain and dysfunction toward its inevitable end. Essentially this leads the victim to be tortured a second time at each stage, because he is left to observe his family’s helplessness in seeing what is transpiring, adding another clever wrinkle to this carefully developed strategy.


There comes a point where the victim and his loved ones plead with the torturer to bring it to its inevitable end, to speed up the clock that is set to bring the drama to its close.  But this falls on deaf ears; that would defeat the whole point.  There is no early exit from the meticulously planned progression.


And yet another wrinkle is being considered, though still under research and subject to the approval by the ethics board under which the torture establishment operates. In the room, which, despite its clinical appearance and ongoing medical support, we literally can term a torture chamber, is a switch that a loved one can pull to speed up the termination. But this is effectuated at the peril of incarceration. So now the drama is confounded: the victim has to absorb the torture to protect his loved one, the loved one is torn to save the victim while sacrificing herself.  

Thursday, October 23, 2014

Ex Ante versus Ex Post Social Policy

I have written various posts on social policy related to the question of whether and how we redistribute income. One argument of equal opportunity and social egalitarianism is that it is fair to control for the randomness of life. For example, take the luck of the draw in who your parents are. If your best career choice was to be born into a wealthy family, if you got ahead because your father could pull strings to get you an internship at a big investment bank; because you built up your high school resume by spending a summer building houses in Tibet while a classmate had to spend the summer working at the 7-11; or because you could afford two years of one-on-one tutoring for the SAT, the result might be to score higher in an objective standard of selection and have a thriving career and high income as a result. But is that objective standard a fair standard, is it really merit based?

I think of income redistribution as an ex post policy. Another approach is to make ex ante adjustments to level the playing field, and then step away and let the chips fall where they may. When properly executed the ex ante approach is consistent with a meritocracy, and indeed creates a better, deeper and more successful meritocracy than ignoring the differences in essential endowments. 

Assume that there is an objective standard for merit, and a test that correctly ranks the subjects in terms of that standard. (For the record, though basing merit on a testing regime is common in many societies, I do not advocate it). Also assume that we can identify the factors that govern success on the test that are within the control of those taking the test, such as how hard they work, as well identify as the factors that are beyond their control. Given these two assumptions, one scheme for the redistribution, suggested by John Roemer (and in this short post I cannot do justice to his argument and stray from it in various respect), is first to define what constitutes the endowment of important characteristics that are outside a person's control, and then assign people to cohorts based on their levels of this endowment.  For example, if the endowment is parents' wealth and parents’ education, we place people into cohorts based on the level of these two factors, with the cohorts made narrow enough so that we can take all those in each cohort as being the same with respect to the endowment.

The social policy is to reward the top one percent of those in each cohort equally, and redistribute as need be in order to do so, and do so for the next percentile across each cohort, and so on all the way down. The idea is that the cohorts take into account differences due to what is beyond a person’s control – the endowments on which the cohorts were based – whereas the person’s place within the cohort is based on those things that are within their control, such as how much effort they expend. That is, by definition a person is not responsible for his cohort, and should not be rewarded or disadvantaged by it, but he is responsible for where he sits in that cohort.  So when we make decisions in terms of social policy, whether it be redistribution of income or equal opportunity, we adjust percentile by percentile across the cohorts.

Redistributing income or other rewards percentile by percentile across the cohorts is an ex post approach to social egalitarianism. It is ex post because we look at results versus the endowment and ask how we can make things fairer given the outcomes, where we have determined that fairness means people should be accountable and rewarded only for those things that are within their control.  

We can use this methodology on an ex ante basis, where we adjust to create equal opportunity rather than equal ex post reward. Applying the cohort approach ex ante is consistent with a merit-based policy.  Doing so requires one more assumption, namely that we only put in the endowment things that are both outside the person’s control and also are improvable if targeted with resources. The aspects of the endowment that we adjust for are those that only make a difference over time and where any reallocation occurs before that difference is significant. This means we do not adjust for innate ability or genetically-based advantage, even though these are things that are outside of one’s control. We do not define the cohorts based on these things, we do not equilibrate these through matching percentile by percentile across the cohorts. 

To give a sense of the ex ante social program which melds social egalitarianism with the objectives of a meritocracy, consider the following, which makes a good story and also happens to relate to an actual situation. A number of ten year-olds are trying out for a spot on the school's tennis team.  There is one spot available, and at the end of the day it comes down to a match between two boys. One has taken tennis lessons for years and practiced many times a week, the other has only played casually every now and then. The well-trained boy wins the match, but only barely. Given his lower endowment of lessons and practice, the loser is a remarkable talent. On the basis of an ex post meritocracy where the established metric was who won the match, the comparatively mediocre boy got the spot.  Yet what is a meritocracy ex post is a mediocrity ex ante. Absent the resources for training and competition the inexperienced boy would receive from being on the team, he goes back to other activities, and the world must make due with one less exceptional tennis player.  We will all survive; but the point repeats itself many times over in fields that are weightier.

If we take the amount of tennis training as the endowment that is outside one’s control, the boy without the past training might be at the top one percent of the bottom cohort, while the boy who beat him might be in the middle range of the top cohort. A merit-based social policy will reallocate resources for training to the inexperienced boy; perhaps a level of resources equal to that which the top one percent of the top cohort is receiving.  If our measures are correct, and if, as assumed above, the reallocation is given early enough in the child’s formative period, we will have someone who is exceptional at tennis who otherwise would have fallen by the wayside. And we will have used those resources more efficiently than if we kept them with the other boy.

The approach I am describing is not the one we are take in the U.S. when we attempt to level the playing field by making adjustments to create equal opportunity. The social program we follow in the U.S. fails in part because it does not distinguish between that which is within and outside of one’s control. It does not even attempt to define cohorts and pick across the various percentiles to reflect effort.  And when it moves into ex ante sphere, it throws resources at differences that are not improvable. (There is one program that might appear to move in this direction: The top ten percent of students from every Texas high school are admitted into the university system. We can think of each school as representing a cohort, with the program giving the same opportunity to those in the top decile of each. But it is pointed in the direction of ex post rather than ex ante social egalitarianism because by the time of high school graduation a student who has had poor preparation is likely to be too far gone for the equal opportunity to yield equal results).


In addition to fostering a fairer society, the ex ante approach described above will yield better results than running people through the merit-based process absent this social policy because there will be a  larger group that is equivalent to the top decile of the top cohort. But no matter how well we execute ex ante social egalitarianism, we cannot get away from addressing it ex post as well. Not everything can be measured, and some things that can be measured cannot be mitigated. At some point we have to say we have done enough; we do not want the end result to be physicians equally drawn from the top of each cohort if, at the end of the day, all of the ameliorative actions have failed to close the gap. The question that remains then is at that point, ex post, do we enact a social policy to redistribute income, or do we consider the social task to be completed?

Thursday, October 16, 2014

My Recent Work on Agent-based Modeling

As is obvious by looking at the time span between recent posts, I have not been active in blogging. I am planning to get back into things.  As a start, to help catch up on what I have been doing, here are a couple of papers and a presentation I recently made on the area of my focus, using agent-based modeling to help assess financial vulnerabilities.

The model is presented here.

For a concise presentation on the model and its application you can look at a talk I did in August at the Newton Institute for Mathematical Sciences at Cambridge University.  (Despite what is suggested by the venue, I did the presentation mathlessly).

Related to the model is work I have been doing to develop a map of sorts for the key agents in the financial system and the ways in which they link to one another. The fist step is what I call the funding map.

Wednesday, May 7, 2014

Piketty Myopia

Thomas Piketty's book Capital in the Twenty-First Century has been lauded for its detailed analysis of data and for the end result of that analysis, a sweeping statement of an inexorable widening of the distribution of wealth between those with capital and those without. There is unabashed glee among those who have bemoaned the plight of the middle class as the one percent has pulled away, waving the book as the gotcha ya for income redistribution.  As should be clear from some of my previous posts, I am one of those people bemoaning the plight, but I am a little slower than many others to take up the Piketty banner.

The book's argument looks at wealth and growth data over the past three centuries, and takes its implications forward with a similar, historical scope.  “In the twenty-first century” is just that; Piketty is willing to push the implications of his analysis not just over the next decade or two, but about as far as you want.  To hear people discuss it, he has discovered the equivalent of a law of physics, namely that capital grows faster than GDP, and if those who own the capital are taking a larger share than GDP has to offer, those without capital must be taking less.  Given the miracle of compounding, this is going to lead the capitalists not only to pull away from the proletariat, but pretty much to own everything.

It is easy to run into problems when you are assuming the future will look the past, all the more so when you are also assuming exponential growth.  Capitalism is not a law of nature; it need not look the same in the future as in the past; it doesn’t even need to be at the core of society. In the sweep of history, capitalism is a recent development, one that happens to encompass roughly the three hundred years Piketty is using for his analysis. So we should temper our excitement; his work is really based on one, very long-term, sample. To see this point, rather than looking back for three hundred years, go back a few thousand.  In the West we have had two other periods comparable in many respects to that of the Industrial Era of today: The Roman Era, and then roughly a thousand years later the Commercial Era. Capital has been the dominating force this time around; in the others it was the military/political complex and the Church, respectively. And in both cases, the writers of the time assumed the structure of the era would go on indefinitely. It was just the way the world worked.  If we went back to the these times, the theme would be the consolidation of the political corpus or of the Church, the analysis would not have been centered on the divisions between those with capital and those without. (I won’t get into the comparable time period in China’s history, but given that Piketty cloaks his thesis with the scope of all human history, he should add a second sample to his analysis and see how his thesis fits the dynasties in China over the past couple of millennium. My bet is, not so well).

Will the future look like the past?
What might move us away from capitalism being the universal underpinning for society? Can the world look different in the future? Of course it can. How that happens and what comes next is an open question – people can't even agree on what led to the end of the Roman Era, and there we have the advantage of history to guide us. But here are some things just to give a flavor of the ways in which the capitalist system might be nudged from the current being the true and final state of mankind:

Maybe the need for capital will diminish. All of the discussion of labor and capital presupposes that labor and capital matter. Will we move into a production mode where our demands are met with low capital expenditure? Piketty recognizes the obvious point that if there is little need for capital – and technology could bring that about – then his argument is no longer valid: “If capital is of no use as a factor of production, then by definition its marginal productivity is zero. In the abstract, one can easily imagine a society in which capital is of no use in the production process.” It might be something to think about in the abstract, but in his view it is not worth lingering, because “in all known human societies, including the most primitive, things have been arranged differently.” That is a pretty broad statement. Even if true, if extrapolating exponential growth is one red flag, a dismissal based on an its-never-happened-before argument is a second.

Maybe we will all become capitalists. Look at the low amount of capital that has been needed for many recent start-ups. Those supplying the labor are also the owners and bring with them their own capital. Perhaps those who labor also be those who own the capital with which they labor? In subsistence agriculture, each farmer owns his own capital, his plot of land. Why not subsistence capitalism? We already have outsourced many tasks to ourselves by using the cheap capital of computers and the Internet; and will be able to do more as 3-D printing and crowd sourcing comes of age.

Maybe the implications of differences in income and wealth will diminish. All of the discussion of wealth and income distribution presupposes that differences in income and wealth matter. As Piketty notes, the nineteenth century novels focused on wealth. Wealth was measured by the number of servants, and thus wealth as a multiple of average earnings x income from capital was key determinant of status. What is the focus today; what might it be twenty or thirty years from now? Looking at the middle class, which is the focus for both Piketty and many of his adherents, could we end up in a world where the differences from a day-to-day lifestyle standpoint are compressed between the super rich and the 'doing OK'? This can happen if what both the super rich and we spend our time on is inexpensive. We are heading in the direction already. We share the same smart phones and spend time in front of the same screen consuming the same content. Of course when it comes to rare wines, contemporary art, and airplane views of Central Park, a gap will remain. But for many of the super rich, we already are seeing erosion; a generation ago cars divided the rich from the middle class; now some of the wealthiest drive the same car I do.

Maybe the consumption of capital-intensive goods will diminish. Consumption drives capitalism. What happens to the consumption of the products of the Industrial Age if we move into a more virtual economy? Consumption of virtual goods almost by definition does not require a lot of the brick and mortar capitalism that has dominated the Industrial Age. Some things, like cell phones and televisions are getting cheaper and cheaper, others like education more and more expensive. If those in the lower rung weight the former more heavily than the latter – and if at the end of the day we find the day-to-day consumption of even the wealthy is also biased that way (I say “at the end of the day” to mean in the post-college years), then much of the discussion of the distribution of wealth becomes moot.

The Exception is the Rule
Piketty notes that the events from 1914 through 1945 altered the course of expanding inequality, and did so even into the 1970’s and 1980’s. “One major lesson is already clear: it was the wars of the twentieth century that, to a large extent, wiped away the past and transformed the structure of inequality.” But that is now the past, and we are back on track:  “Today, in the second decade of the twenty-first century, inequalities of wealth that had supposedly disappeared are close to regaining or even surpassing their historical highs.”

The period of retrenchment from his law of growing inequality is hard to take as an aberration given that it extends nearly seventy years. That is a big chunk of time, even if you are looking at a period from the 1700’s to the present. But Piketty does pull out this period as an aberration, as an exception to the rule. But it is not. It is part of the rule. It is an example of the ebb and flow of society and nature, of the noise that destroys and supplants family fortunes, states, and civilizations. It is not unique: for example, in the fourteenth century we had the Black Death, the Hundred Years’ War, and years of devastating famine. Not to mention financial failures that dwarf those of the Great Depression – but they in turn were so dwarfed by the other calamities that they hardly merit a footnote for the period.

Piketty is unequivocal about the effects in the early twentieth century being an aberration, but not about why it was a period when his thesis did not obtain. “…we must insist on the fact that the fall in the capital/income ratio between 1914 and 1945 is explained to only a limited extent by the physical destruction of capital (buildings, factories, infrastructure, etc.) due to the two world wars…In fact, the budgetary and political shocks of two wars proved far more destructive to capital than combat itself.”

So it was not just the destruction of capital, but political shocks. But if that, here is another idea: The period between 1914 and 1980 was the only extended period from the ushering in of the Industrial Age to now where there was no industrial revolution. The first industrial revolution ran from about 1760 to 1840.  The second went from the latter half of the 19th century (around the time of the introduction of Bessemer steel in the 1860s) until World Was I. And the third industrial revolution, the computer age, began to gather steam in the 1980s. So I could take Piketty's data and argue that the disparity occurs during periods of industrial revolution, and falters otherwise. No industrial revolution, no problem.

But I'm not going to move into a teleological discussion on how our society could be transformed, anymore than I would talk about the demise of the Roman or the Commercial Era, except to point out the obvious, that unexpected things come along. And to note that, as I have pointed out elsewhere for the financial markets, the more complex and tightly coupled a society or era, the more likely something that does come along can exact a fatal toll.

Conclusions are Opinions
We can move on from Piketty’s thesis to his policy conclusions, and do so independent of the arguments that make up the bulk of the book. It is enough to have the opinion, as many already do, that the disparity in income and wealth is too great, and that the immobility up and down the ladder also is too great.  Or that, absent political action, these will become too great in the future. What disparity and immobility is too great is a matter of opinion, and after his analysis, that is how it remains. And if the issue is that the disparity will become too great in the future, the prudent course might be to wait and see what the future brings. The question of redistribution is a political, even a philosophical issue, not an economic one. (A side note: He differs from some others in arguing for taxes on wealth, but that is nothing new. Indeed that is how things were taxed in the China’s Song Dynasty).

Piketty’s opinion seems to be that if we project out to the point that the wealth gap becomes what it was during the Belle Epoque of the early 1900's, we have gone too far. Another person could argue that such a disparity is fine, and then we are at the “agree to disagree” point of the argument.  He does ask, “Taking all these elements into account, what is the “right” split between capital and labor? Can we be sure that an economy based on the “free market” and private property always and everywhere leads to an optimal division, as if by magic?” There are not many who really believe in that magic. The question of the ideal split between labor and capital has been debated for over two centuries, which is to say it is a question that will probably never arrive at an answer. Marx stated it as the center of class warfare, couching it in terms of the working day:

The capitalist maintains his rights as a purchaser when he tries to make the working-day as long as possible…. On the other hand...the laborer maintains his right as seller when he wishes to reduce the working-day to one of definite normal duration. There is here, therefore, an antinomy, right against right, both equally bearing the seal of the law of exchanges. Between equal rights force decides. Hence is it that in the history of capitalist production, the determination of what is a working-day, presents itself as the result of a struggle, a struggle between collective capital, i.e., the class of capitalists, and collective labour, i.e., the working-class.

Marx begins with an acknowledgement of the perception of rights on the part of both the capitalist and the laborer, but then argues that the question of the length of the working day cannot be solved by an appeal to rights, but only through class struggle, wherein “force” decides between “equal rights”. (Force can mean physical force, but can also mean the force of the political process).

It is difficult to do much more than simply take sides when it comes to economic rights, because what we call economic rights are really nothing more than the bargaining in an exchange between those providing labor and those providing capital. The social and economic pie has to be split, and there is no objective way to do so. There are some areas of fairness in the civil sphere – freedom from slavery and torture – but what are the rights inherent for a particular term of exchange between the parties in a trade?

Where does this battle lead? For Marx, left unfettered capital digs it own grave, for Piketty, it grows without bound and swallows the world. For Marx, the return on capital falls to zero, for Piketty it persists above the growth rate of the economy. We can argue that Marx is wrong because up to this point we have not witnessed his outcome. I argue that Piketty is also wrong, because he makes the common error of assuming the future will look like the past, even as he recognizes that the past holds a source of failure for his thesis.



Views and opinions expressed are those of the authors and do not necessarily represent official OFR or Treasury positions or policy. 

Tuesday, March 18, 2014

Big Data versus the SAT


In a recent Time Magazine article, the president of Bard College, Leon Botstein, joined a chorus of criticism of the SAT, going so far as to call it “part hoax and part fraud.”  Criticism is coming fast and furious because the SAT has just unveiled a new, improved product to try to fend off a trend among competitive colleges to downplay the role of the SAT, and even to eliminate its use entirely.  (The not-for-profit status of the College Board, which produces the SAT, does not put the company beyond reacting to a profit motive; not-for-profit does not exactly mean you don’t get benefits from pulling in more revenue).

Among the critiques are that performance during one afternoon in the junior year of high school should not govern a student’s future (the SAT is not only used for college admission, but also by employers further down the road); that the SAT has little to do with what really constitutes learning and productivity (I haven’t done anything useful in my work by pulling the right answer off of a multiple choice test for as long as I can remember); that it correlates with income more than it does with anything else; and that what has now become years of SAT preparation detracts from more productive learning.

And that is just the start of the list. By its very nature as a standardized test, to the extent colleges rely on the SAT they are looking at exactly the same criteria, so the same sort of students will percolate to the top of the stack. The creative and off-the-wall student who adds the richness and intellectual diversity that a college seeks might be blown off course because there is no reason to think that such a student will do well in a timed, multiple choice test – or even that such a student will have an interest in the many hours of preparation courses required to be competitive in the test. To take the effect of the SAT exam on creativity to the extreme, read a previous blog post I wrote on the effect standardized testing has had on creativity in Asian countries.

You would think that in the emerging world of big data, where Amazon has gone from recommending books to predicting what your next purchase will be, we should be able to find ways to predict how well a student will do in college, and more than that, predict the colleges where he will thrive and reach his potential.  Colleges have a rich database at their disposal: high school transcripts, socio-economic data such as household income and family educational background, recommendations and the extra-curricular activities of every applicant, and data on performance ex post for those who have attended. For many universities, this is a database that encompasses hundreds of thousands of students.

There are differences from one high school to the next, and the sample a college has from any one high school might be sparse, but high schools and school districts can augment the data with further detail, so that the database can extend beyond those who have applied. And the data available to the colleges can be expanded by orders of magnitude if students agree to share their admission data and their college performance on an anonymized basis. There already are common applications forms used by many schools, so as far as admission data goes, this requires little more than adding an agreement in the college applications to share data; the sort of agreement we already make with Facebook or Google.


The end result, achievable in a few years, is a vast database of high school performance, drilling down to the specific high school, coupled with the colleges where each student applied, was accepted and attended, along with subsequent college performance. Of course, the nature of big data is that it is data, so students are still converted into numerical representations.  But these will cover many dimensions, and those dimensions will better reflect what the students actually do. Each college can approach and analyze the data differently to focus on what they care about.  It is the end of the SAT version of standardization. Colleges can still follow up with interviews, campus tours, and reviews of musical performances, articles, videos of sports, and the like.  But they will have a much better filter in place as they do so.