Jeff Duntemann's Contrapositive Diary Rotating Header Image

The All-Volunteer Federated Encyclopedia of (Really!) Absolutely Everything

My regular readers will recall that I wrote an article in the June/July 1994 issue of PC Techniques, describing a distributed virtual encyclopedia that pretty much predicted Wikipedia’s function, if not the details of its implementation. My discontent with Wikipedia is not only well-known but not specific to me: The organization has become political, and editor zealots have various tricks to make their ideological opponents either look bad, or disappear altogether. Key here is their concept of notability, which is Wikipedia’s universal excuse for excluding the organization’s ideological opponents from coverage.

In one of the decade’s great hacks, Vox Day created Infogalactic, which is a separate instance of the MediaWiki software underlying Wikipedia and a fair number of other, more specialized encyclopedias. Infogalactic has a lot of its own articles. However, when a user searches for something that is not already in the Infogalactic database, Infogalactic passes the search along to Wikipedia, and then displays the returned results. I don’t know whether or to what extent Infogalactic keeps results from Wikipedia on its own servers. It’s completely legal to do so, and they may have a system that keeps track of frequent searches and maintains frequently searched-for Wikipedia pages in local storage. Or they may just keep them all. We have no way to know.

Infogalactic’s relationship with Wikipedia immediately suggested a form of federation to me, though Infogalactic does not use that term. (Federation means a peer-to-peer network of nodes that are independently hosted and maintained yet query one another.) The Mastodon social network system is the best example of online federation that I could offer. (It’s not shaped like an encyclopedia, so don’t take the comparison too far.) There is something else called the Fediverse, which I have not investigated closely. In a sense, the Fediverse is meta-federation, as it federates already federated platforms like Mastodon. For that matter, Usenet is also a form of federation. It’s been around a long time.

The MediaWiki software is open-source and freely available to anyone. There are a lot of special-interest wikis online. One is about Lego. (Brickipedia, heh.) For that matter, there’s one about Mega Bloks. Hortipedia is about gardening and plants generally. It’s a huge list; give it a scan. You might find something useful.

My suggestion is this: Devise a MediaWiki mod like Infogalactic’s, but take it farther. Have a “federation panel” that allows the creation of lists of MediaWiki instances for searches falling outside the local instance. A list would generally start with the local instance. It might then search instances focusing on related topics. The last item on most lists would be a full general encyclopedia like Wikipedia or Infogalactic.

Here’s a simple example, which could defeat Wikipedia’s notability fetish for biographies and a lot of other things: Begin a search for a given person (or other topic) with Infogalactic, which, remember, searches Wikipedia if its own database doesn’t satisfy the query. So if that search fails, submit the same search to EverybodyWiki, which doesn’t apply notability criteria to biographies. In fact, EverybodyWiki does what I suggested be done a number of years ago: It collects articles marked for deletion on Wikipedia, of which it currently has over 100,000. I’m tempted to post a biography on Wikipedia just to see if, when it’s deleted (and it would be) EverybodyWiki picks it up.

(As an aside: I just found EverybodyWiki a month or so ago, and I’m surprised I hadn’t heard of it before. It has more than just biographies and is definitely worth a little poking-around time.)

Now, the tough part: How would this be accomplished? I don’t know enough about MediaWiki internals to attempt it myself. There’s an API, and I’ve been surfing through the API doc. There’s even an API sandbox, which is a cool idea all by itself. Alas, there are remarkably few technical books on MediaWiki, and the ones I would be most interested in get terrible reviews. Given how important MediaWiki is, I don’t understand why tech publishers have skated past it. My guess is that few people bother to do more than custom-skin MediaWiki. (There’s a book on that, at least.) If the demand were there, the books would probably happen. If you know enough about MediaWiki modding, I’ll bet you could find a publisher.

I’m thinking about installing MediaWiki on my hosting services, just to poke at and try things on. Hell, I predicted this thing. I should at least know my way around it.

If you’ve done any hacking on MediaWiki, let me know how you learned its internals and what you did, and if there are any instructional websites or videos that I may not have encountered.


  1. I remember you writing about that. I just pulled my copy of June/July 1994 PC Techniques off the bookshelf to re-read it. I miss PC Techniques… it was such a great magazine. It’s the only magazine I read cover to cover as soon as I received it. Good times.

    Is there anyway you could make PCT and VD available as pdf downloads?

  2. TRX says:

    Some of Jeff’s bits are here:

    The last issue was over twenty years ago. (time flies, and all that…) Is there a rights problem, or do they just need someone to scan in old issues? I’d be willing to donate some labor with ye aulde scannere.

    1. There are nominal rights issues. I bought only first rights for the articles, and it’s a point of honor for me to treat my authors fairly. The ones I posted at the link you shared were all given with explicit permission. Posting whole issues is probably not an option, since each issue probably contained one or two items by people I can’t find in the present day.

  3. //Infogalactic has a lot of its own articles. However, when a user searches for something that is not already in the Infogalactic database, Infogalactic passes the search along to Wikipedia, and then displays the returned results. //

    I’m fairly certain that Infogalactic does not do that (although that may have been an ambition and the idea is neat). The admins manual import pages from Wikipedia into their version.

    For example, take this page from Wikipedia:
    If you put that entry into Infogalatic you’ll get the standard wiki response for a page that doesn’t exist yet.

    1. Well, I ran your test, and sunuvugun: The page you cite did not exist on Infogalactic. I use Infogalactic a lot, and this is the first time I’ve had a search for a known-to-exist Wikipedia article fail. Now, the article itself contains very little information, and I wonder if it was tossed by Infogalactic for being, well, close to empty. (I also question whether the subject of the piece is in fact notable enough by Wikipedia’s own standards for inclusion on the site.)

      All I know about Infogalactic internals is what I read from the source. I asked someone I know who’s an editor over there and he had no idea how the thing works under the surface.

      The underlying MediaWiki software has an API, and having flipped through it I’m pretty sure that something like this is possible. If I had to guess, it may be that kicking a search query over to Wikipedia took so much time that the site looked bad to casual users. Hard drive space being as cheap as it is, maybe it was easier to just grab the articles and store them locally.

      So. Thanks for letting me know. This can be a crazy business sometimes.

Leave a Reply

Your email address will not be published. Required fields are marked *