Big Fractal Tangle: getting the semweb exactly wrong

Reading "Ambient Findability", I came across Peter Morville's discussion of the Semantic Web, which references Shirky's lame criticism of it ("The Semantic Web is a machine for creating syllogisms").

He also quotes David Weinberger:

I fear that the Semantic Web will go the way of SGML and for basically the same reason: normalization of metadata works real well in confined applications where the payoff is high, control is centralized and discipline can be enforced. In other words: not the Web.

Reading such comments confounds me, since they've got it *exactly* wrong. The Semantic Web approach is LOOSE, not normalized. The beauty of RDF and OWL is that you can take messy incompatible schemas and interconnect them after the fact, giving structure to what was once incompatible. The semweb community has always taken a "build them as islands, then stitch them together" approach, as they really do understand the need for looseness.

How did this happen? Why is the Semantic Web so misunderstood by so many? Here's my guess: In the beginning, lot's of really smart semweb researchers looked at the Web and said, "What a mess. How can we tie this all together in a webby way that lets us solve real problems?" RDF and OWL were born.

As the Semantic Web grew, much of the talk was of ontologies. Given the very specific nature of most of the early problem domains (medical, etc), an outsider could easily see all of it as too *disciplined*. People like Shirky and Weinberger came into the middle of the discussion, when everyone was talking about how to make it rigorous. They missed the early part when talk was about how to make it loose.

What didn't help is that most of the talking was being done by standards body formalists, who live and breath structure and specification to a degree the likes of which only dictionary writers can appreciate. The outsiders listened in and heard extremely-specific-talk, and made the assumption that these formalists were making something which was too confining for the Web. This is the tragedy: the pundits think the semweb is brittle, when it's exactly the opposite. It's the very thing we need to add structure without losing looseness.

So, here we are, watching terms like "Web 2.0" and "folksonomies" get traction instead, simply because pundits (who can write clearly) understand them more easily, and don't need to know Backus-Naur Form to grok what's going on. RDF and OWL serve the same purpose, but most of the semweb work now being done is in more rigorous domains, so it seems like it's only appropriate to academics and large monolithic software efforts. So we're stuck with loose semi-structured solutions instead of semweb technologies.

At some point, the Web 2.0 crowd are gonna hit the same wall the semwebbers hit a while back, which lead to the RDF/OWL approach in the first place. They're gonna say, "What a mess. We've got all this metadata and it's an incredible pain to do anything with it." Maybe then, the Shirky's of the world will say, "Ah ha! The Semantic Web isn't about syllogisms, it's about STITCHING."

But by then it'll be too late, so my renewed goal for 2006 is to continue explaining the virtues of semweb technologies to non-formalists, before the world gets saddled with some folksy half-baked semi-standard that does half of what RDF and OWL do, but in a way that makes it impossible to do anything useful down the road.

As I said at the start of this blog, we need a better whirlpool rap.

I think there are probably several things that have gone wrong, leading people like me (or just me) to think of the semweb as primarily a way of tightening up the semantic nuts on the Web. (I'm using "nuts" in its literal sense here! :) For one thing, the semweb has had some mood swings. For another, it has attracted a wide range of supporters who come at it from different backgrounds and dispositions. Plus, there are various levels and meanings of "loose." Then there are several tempting conflations, such as "loose" with "bottom up." And more.

It'd thus be extraorinary helpful to me to get clearer about the ways in which the semweb loosens vs. tightens, imposes vs. is shaped by, drives from the top vs. from the bottom, normalizes vs. clusters, etc. Wanna help me not be as stupid in 2006 as I was in 2005? Let's talk...

posted by David Weinberger at January 4, 2006 03:15 PM

Reading such comments confounds me, since they've got it *exactly* wrong. The Semantic Web approach is LOOSE, not normalized. The beauty of RDF and OWL is that you can take messy incompatible schemas and interconnect them after the fact, giving structure to what was once incompatible. The semweb community has always taken a "build them as islands, then stitch them together" approach, as they really do understand the need for looseness.

The problem is that the above is simply wrong. Ontologies are exact, have different domains and are designed to be so. There is no guaranteed correct way to "stitch" two different ontologies together [for one thing, because their domains usually differ] and derive anything other than one of many possible conflicting uber-ontologies. You can pretend that it works but it won't; you can go through the motions and succeed in special cases, but other cases will fail. The result is equivocation and confusion.

Standards bodies should not be created for a nonexistant technology. The "semantic web" exists as a vague concept but there is no corresponding well-defined semantic web technology that works as you describe. Sadly no one wants to pop TBL in the head and tell him to face reality since he is such a celebrity. But the W3C body is doomed and at some time he should acknowledge what's happened.

posted by glardo at January 5, 2006 09:28 AM

TimF, good article.

Glardo talks of the Semantic Web as "vague concept" when there is an increasing number of specs out there and in use. The Sem Web stack roadmap has been around for many years, and has been (in large granularity) stable, and the layers have been deevloped and standardized one after the other.

I guess the whole idea of having local consistency but no global consistency in a system of interconnected documents is a pardigm shift. It is difficult to understand. Like the WWW, you can't describe it in pre-web concepts. It isn't the push for one monolithin ontology. It isn't a fuzzying up of logic so that inconsistencies don't exist. It is a fractal system on some global, some local concepts and all scales in between. I talked about it this at ISWC in Galway, but haven't written it up in detail.
http://www.w3.org/DesignIssues/Fractal is sort of in the area.

I remember people who were just as vocal about the WWW, saying that the lack of overall structure would just be a total mess, and that the possibility of dangling links would be intolerable. They underestimated the benefit of connectivity, the incentive to become connected, and their ability to put up with a (valuable) mess.

posted by Tim Berners-Lee at January 5, 2006 05:13 PM

Thanks. Yes. It was your comments regarding this at ISWC in Sanibel that led to the name of this blog :)

I've struggled over the last few years trying to describe this dual nature in layperson terms, and have a handful of phrases I return to ... "there is no box" (when talking to object-oriented folks, trying to get them to stop thinking in objects) or "the pick-up sticks" nature of triples, as though each triple were a popsicle stick, and "classes" were a toy log cabin made from them. You can pull apart the log cabin and make a new one with the sticks and there's no information lost... you can even make ten log cabins (at the same time) with the same sticks. Too often, it leaves people scratching their heads and not sure where to start.

It's similar to the paradigm shift of object-orientation. Simple enough to describe, but it requires more hand-holding than might be apparent at first.

I'm betting there's a decent emerging market in "semweb mentoring" because of this, similar to the OO mentoring market. Such street-level mentoring sessions could prove very useful to the adoption of semweb techniques.

posted by Timothy Falconer at January 6, 2006 12:48 AM