Taleb's "The Black Swan" is a recently read book that describes in great detail one of the underlying phenomenom of the current credit crisis (via Gregor):
Our brains are wired for narrative, not statistical uncertainty. And so we tell ourselves simple stories to explain complex thing we don't--and, most importantly, can't--know. The truth is that we have no idea why stock markets go up or down on any given day, and whatever reason we give is sure to be grossly simplified, if not flat out wrong.
Nassim Nicholas Taleb first made this argument in Fooled by Randomness, an engaging look at the history and reasons for our predilection for self-deception when it comes to statistics. Now, in The Black Swan: the Impact of the Highly Improbable, he focuses on that most dismal of sciences, predicting the future. Forecasting is not just at the heart of Wall Street, but its something each of us does every time we make an insurance payment or strap on a seat belt.
The problem, Nassim explains, is that we place too much weight on the odds that past events will repeat (diligently trying to follow the path of the "millionaire next door," when unrepeatable chance is a better explanation). Instead, the really important events are rare and unpredictable. He calls them Black Swans, which is a reference to a 17th century philosophical thought experiment. In Europe all anyone had ever seen were white swans; indeed, "all swans are white" had long been used as the standard example of a scientific truth. So what was the chance of seeing a black one? Impossible to calculate, or at least they were until 1697, when explorers found Cygnus atratus in Australia.
I believe we learned everything we needed to prepare for the current crisis with the Long Term Capital Mgmt failure (risk model failure during market-wide sell-off; fund too big to fail thanks to positions in complicated unregulated derivates, failing; govt bailout) which the book discusses at length. And we have addressed none of the risks behind network failures in the intervening years, as interconnections have grown.
Here's hoping that the fix (and I don't mean the bailout) addresses systemic issues this time around and we don't just bury our head in the sand like we did last time, pretending to deal/regulate 1st order relationships while ignoring 2nd order+ effects of the financial web.
Update 1:
a better quote from the actual book...
"Globalization creates interlocking fragility, while reducing volatility and giving the appearance of stability. In other words it creates devastating Black Swans. We have never lived before under the threat of a global collapse. Financial Institutions have been merging into a smaller number of very large banks. Almost all banks are interrelated. So the financial ecology is swelling into gigantic, incestuous, bureaucratic banks when one fails, they all fall. The increased concentration among banks seems to have the effect of making financial crisis less likely, but when they happen they are more global in scale and hit us very hard. We have moved from a diversified ecology of small banks, with varied lending policies, to a more homogeneous framework of firms that all resemble one another. True, we now have fewer failures, but when they occur .I shiver at the thought."
I really love the quicksearch function in the latest google desktop:
"The fastest way to search the web, and your own computer, is to use the Quick Search Box in the center of your desktop (simply press the "Ctrl" button twice to call up the search box, and press "Ctrl" twice again to hide it). Just type a few letters or words into the search box and your top results pop up instantly. You can also use it to launch applications without having to surf the Start menu; for example, you can launch Microsoft Word by typing "wor" into the Quick Search Box and selecting "Microsoft Word" in the list of results that appears. Or, simply type in your search term and press "Enter" to search the web."
I realize app-launchers is not a new thing, but I stay away from utilities like that because I switch desktops too often : it's annoying to rely too much on any customizations or desktop tweaks and few stand the test of time. Google desktop's search-web+gmail+desktop at the same time is too damn useful, but letting my launch apps speeds things up a lot.
[WindowsKey][WindowsKey]"CPAN" -- install new perl module using Strawberry perl cpan shell
[WindowsKey][WindowsKey]"Quicken" -- enter check from client etc.
8:49:50 AM comment []trackback []
Jonathan Ellis writes something suspiciously similar to what I think I would write about App Engine (but he actually went through the trouble of developing with it):
Having been eyeball deep in App Engine for a while, I've reluctantly concluded that I don't like it. I want to like it, since it's a great poster child for Python. And there are some bright spots, like the dirt-simple integration with google accounts. But it's so very very primitive in so many ways. Not just the missing features, or the "you can use any web framework you like, as long as it's django" attitude, but primarily a lot of the existing API is just so very primitive.
I can't get past the fact that there's no SQL. I know why there's no SQL, (Brewer's Conjecture etc) and I have some experience with it. I've been doing years of mostly-simple mashup stuff with quickbase, which is a similar PaaS-backing non-relational data store. It has it's place, but it's hard to imagine really cranking out something complicated with it. When I do code reviews for clients, the biggest improvements come from transforming procedural cursor logic back into set-oriented db-logic, and AppEngine wants you to go the opposite direction. Sigh.
Looking at the bigger picture, I'm trying to figure out what the drivers for PaaS adoption will be, especially in the case where the flavor of PaaS requires retiring your db muscles. Is giving the ability for developers (or non-developers) to clone and enhance code without sweating enough? Maybe there's a non-trivial demand for this (suggested by IBM's Project Zero, or stretching it a bit, MS-Sharepoint), but it seems like a long-term bet.
Aside: a cool App Engine project is Ben Adida's open source, cryptographically secure, online voting project Helios. You can conduct auditable elections and make sure they election-holders counted your vote the way you wanted. The auditing can be done by you personally, or a group that you trust. This all happens while hiding the actual candidate or choice you voted for, so you can't be coerced or sell it. Go Ben! The world needs Helios (but I'd be happy to start with just Florida).
7:27:53 AM comment []trackback []
they just got around to this 8 months ago, and they're fscking Facebook (which means you can wait)
they're still doing all writes to a single datacenter
they're hacking an object-level mark/sweep into the MySQL replication stream suggesting a certain parable of a hammer and nails.
Lol. Takeaway #3 is my favorite.
Regarding the other two takeaways, I think Moore's Law will ensure the simplicity of single-datacenter and/or onsite deployments will be a compelling alternative to the clouds for quite some time.
In other words, use Google App Engine et al if you like the cost and API's. For the distributed scalability, YNGNI.
9:31:48 AM comment []trackback []
The Paradox of Choice is a book with a thesis that's stuck in my head since I read about it a few years back. Simply put, more choice doesn't necessarily make us happier. The video and book gives some hard-to-dispute examples and data to back this up.
The thought that chases me is how much of technological progress is done in pursuit of choice/options/etc, without, as the book makes clear, accounting for the aggregate tragedy-of-the-commons cost of these options. The paradox is that someone somewhere will benefit (immensely?) from the choices technology produces, and there's no easy way to say when enough is enough.
Should we go back to 4-5 channels of TV? Laughable, right? Is it possible we'd be happier if we did? Hmmmm. That's just one way the book starts messing with your head.
The book repeats the central idea many times ... it's a simple enough that you get it well before the end, but I still found it worthwhile to ponder for the duration of the book. I would suggest that you pick up a copy, but that would only be adding to your reading choices (see how evil it is?)
Hey John, this sounds good. For dvds, cds and paperbacks, zunafish.com works well (albeit not quite so karmic). For $1 plus $2 postage, the site sets up swaps. I own about 20 dvds, and I gradually swap them after I watch them, so that about 3/4 are ones I haven't seen. It's like netflix, with a bit less choice online but more DVDs at home and no fixed fees. Now if I could only acquire the time to watch everything I'd like to watch... #
I like the idea ... after getting over the initial Netflix rush during which the near-infinite film choice collided with finite movie watching time, I opted out and tried dvdswap.com. The experiment didn't last long, mainly because I had a big problem with liquidity. At the time, there just weren't enough swaps to be had. Now I just buy on half.com and resell on ebay. The buy/sell spread is usually about $4/movie (includes postage), which is the price I pay at my local video store, except no late fees, costs you nothing if you can't get to it, and no monthly commitment. I use Half.com because bidding on ebay is too much work and the selection of used inventory is pretty good.
Anyway, Bookmooch is incredibly liquid - I think the inventory is > 2 Million books and growing very rapidly. I think 'bigger than amazon' is definitely a possibility.
4:46:58 PM comment []trackback []
I'm enjoying the bookmooch service quite a bit. It's kind of an honor system based inter-library loan, except you aren't loaning books, you own them free and clear. And it's not between libraries, but individuals. But other than that, it's exactly like inter-library loan. :-)
Here's how it works: You put in a list of books gathering dust on your shelf, and a list of books you'd like to read. When there's a match between have's and want's, bookmooch shoots you an email to send or receive the book. You pay postage on outgoing, but the incoming books are free.
It's a treat to have a free book show up in your mailbox ... netflix has that same psychological edge over the local video store, except in the case of netflix they only want you to think it's free.
They keep an upload/download ratio just like the BBS's of yore, but I haven't quite figured out the reputation mgmt system yet. I do know, however, that you can donate your accrued book-donating karma to hospitals so they can get more book donations for patients. How cool is that?
So far I've sent 8 and received three, and I recommend the service to everyone I get within earshot (apologies to friends). Give it a shot.
4:24:43 PM comment []trackback []
My friend Andy launched a startup about 6 months ago, and has been getting lots of press in the last week. He's a bike nut, so that's always what we talk about when I see him : the crazy cost of the big dig and the missed opportunity for public transit, the cities without cars exhibit at the MIT Museum, Boston's legendary bike-unfriendliness, wouldn't it be great to work as a bike messenger instead of behind a desk, yada yada. He decided to make a business from his passion for bikes, and launched the New Amsterdam Project, a bicycle delivery service.
I confess that I was skeptical about the business plan when he described it to me a year ago or so. Somehow I thought he would be doing food delivery with bike messenger overhead ( the bastard son of Kozmo and Webvan perhaps? ). But I'm glad to see that's completely wrong.
It clicked for me when Andy mentioned, "We're really a trucking company" and described some of his first customers: delivering local produce to restaurants and farm-share subscribers. It's all scheduled recurring delivers, mostly business-to-business. He replaces trucks with bikes - as simple as that.
Anyway, I get it now. The zero-carbon-footprint approach to deliveries, making local produce workable, taking cars/trucks off the streets -- it's all incredibly timely with gasoline breaking the $4/gal limit. The MSM has picked up on it : he was on NPR Morning Edition last week and one or more 24 hour cable news networks sometime in the next week or so. And with every media mention, more businesses realize they can somewhat painlessly redirect necessary and frequent spending in an environmentally friendly way.
I believe we can make cities look very different than they do now, and make them work better for the cars and people whose coexistence is quite strained. Some of the change will be driven by technology, but a lot can happen with old-school determination and guts. Andy has the latter, and I wish him luck.
Taleb's "The Black Swan" is a recently read book that describes in great detail one of the underlying phenomenom of the current credit crisis (via 
My friend Andy launched a startup about 6 months ago, and has been getting lots of press in the last week. He's a bike nut, so that's always what we talk about when I see him : the crazy cost of the big dig and the missed opportunity for public transit, the cities without cars exhibit at the MIT Museum, Boston's legendary bike-unfriendliness, wouldn't it be great to work as a bike messenger instead of behind a desk, yada yada. He decided to make a business from his passion for bikes, and launched the 
