reb00ted

We can sustainably, fairly, do better. Let's go.

Tech

By Johannes Ernst

https://reb00ted.org/tech/

  • 2023-04-02

    The architecture of the Fediverse: network, apps and people

    Since Elon’s acquisition of Twitter, the “Fediverse” has become a thing. A thing that many people talk about, often in nebolous terms. I think we need to be more precise, and to do that, it might be helpful to distinguish several parts of the Fediverse.

    I believe the primary components of the Fediverse are the following, best shown as a layered architecture:

    People

    You and me; sometimes companies

    Apps

    e.g. Mastodon, PeerTube

    Network

    ActivityPub (and related protocols)

    To summarize this diagram: there is a communications network (at the bottom), over which apps can communicate with each other (middle), enabling people to interact with each other in various ways (top).

    Have you noticed that when people talk about the Fediverse …

    • … sometimes they mean the network, as in: “all these apps can talk to each other because it’s all ActivityPub” …
    • … and sometimes they mean people, as in: “We have genuine relationships with each other in the Fediverse because no 3rd party inserts ads and addictive algorithms”?
    • And of course, when they use “Mastodon” as the term instead of “Fediverse”, that’s because in this view, the Fediverse is primary formed by thousands of app instances, most of which today are Mastodon instances.

    This distinction is very important, because the properties of the “Fediverse” in the various definitions, are quite different. For example:

    • If I think the Fediverse is all about people, and how they relate to each other, I don’t really care whether it’s based on ActivityPub, or other protocols (like SSB, or Bluesky, or Webmention, or …) or any combination. In this view, the Fediverse is a “place” where certain social norms are different, including that certain behaviors prevalent in society or in other social networks (like expressions of racism) are frowned upon or grounds for blocking.

    • If I think the Fediverse is the set of all software nodes that communicate over ActivityPub, then I’m baffled why anybody should think some of those nodes shouldn’t be bots, or publish advertisements or special offers; ActivityPub can carry all of those and more. In this view, in its extreme form, the Fediverse could ultimately encompass a lot of internet communication infrastructure, including replacing SMTP for “e-mail”, for example.

    Because of this confusion, sometimes the term “Fediverse” is actually less than helpful. On the other hand, there isn’t an obviously better term (or several; we need several).

    In my personal view, the network/interop-centric definition of Fediverse is more useful: the Fediverse is the set of all nodes that can communicate semantically with each other through common protocols (ActivityPub).

    Given this, it allows many new apps to be built that connect to the same network – the Fediverse – creating lots of new value, as I described earlier.

    And those in turn enable people with common values to find each other, and form communities, that practice forms of interaction that would be much harder on centralized social media platforms. To use but one example, my primary Fediverse account is hosted by social.coop, a member-owned cooperative that formed for the purpose of hosting a social media server that works for them; self-funded, self-governed. Imagine that outside of the Fediverse!

    But regardless of what the term “Fediverse” is ultimately going to end up meaning, whether that’s my preference or some other interpretation, let’s not conflate these different ideas, otherwise all the beautiful opportunities created by the Fediverse to communicate with each other won’t help us avoid talking past each other about the very thing that makes it possible to talk!

  • 2023-03-25

    My wish list for future ActivityPub standardization and related activities in the Fediverse “commons”

    Note: Newer version of this list is here.

    There is some movement in the W3C to perhaps reactivate working on ActivityPub, the protocol that lets Fediverse apps such as Mastodon talk to each other and form a large, decentralized, social network.

    I posted this to their mailing list recently. Sorry if this is a bit cryptic, it was written for a very specific technical audience. I have added a few annotations in square brackets to provide context.

    What am I missing in my wish list?

    Putting on my product developer hat, here’s what I want for Christmas:

    A single-document basic [Fediverse] interop spec, ie.

    • when I have written code for everything it says, my code will interoperate in some basic fashion with all other software that has implemented this document
    • no need to consult or understand other implementations
    • may be quite basic, e.g. text content only, only a minimal set of activity types
    • enables implementors to have a “MVP”-style interop at lowest-possible engineering effort (including the time required to read/understand the specs!)
    • This could be done as a “minimal profile” of a stack that contains a subset of AP [ActivityPub], AS [ActivityStreams], and Webfinger

    A test suite for that profile

    • suitable to add to my automated test suite
    • over time, this test suite can grow beyond the minimal profile

    A branding program for products that have passed the test suite

    • As an implementor, you get to put the sticker on your product.
    • In particular, in the places in the product where users “connect” to other servers in the Fediverse, like “Visa” is displayed at the POS terminal
    • I believe this will become critical if/when larger orgs with potentially different value systems connect to the Fediverse

    A set of web “intent buttons” for Like, Follow, Post, etc that work across sites

    • like they exist for centralized social media
    • as easy to use for the user
    • we can argue how this can be accomplished technically. I have opinions, but for this wish list, they are immaterial, as long as I get those buttons :-)

    A standardized way to express terms under which content is provided

    • As I understand it, Bob Wyman calls that a Rights Expression Language
    • This probably should start with a use case collection

    A design for search that meets the requirements of all relevant parties and points of view

    • This is probably far less a technical problem than one of successful communication

    A design to reduce certain loads

    • Fan-out
    • Video
    • (Not my area of expertise, so I don’t have details)

    Improved identity management across the Fediverse

    • Easy-to-use single-sign-on across servers. Use case: I use several apps for different content types (like micro blog and video). Bonus: they all post from the same identifier
    • Easy-to-use persona management. Use case: I have a personal and a work account, bonus if they can be on the same server
    • Identifiers not tied to the domain name system

    Some of this squarely falls in the domain of this group [that would be the W3C’s Social Web Interest Community Group], some is adjacent. It could be pulled in, or it can be done somewhere else. I don’t particularly care about that either, as long as it gets done and done consistently with the rest.

    Now I’m sure you all are going to tell me why I can’t have all those things for Christmas, and certainly not this Christmas. But I can wish, no? (More seriously, I think they are all essential for continued successful growth of the ActivityPub network as new parties connect)

  • 2023-03-22

    Open networks are more valuable than closed ones: the case for the Fediverse over Twitter and Facebook

    Networks are everywhere, and they are valuable. Consider:

    • The road network. Imagine you’d need a different car for each piece of road. Life as we know it today would be utterly impossible without it.
    • The phone network. To pick two random use cases, without it you couldn’t call customer service or summon an ambulance.
    • The Visa credit card network (and its competitors). You would have to use cash instead, but arguably everybody accepting the same currency forms a network, too, and without that, we’d be back to barter. Which would be really inconvenient.
    • The world-wide-web. Some of us are old enough to remember the times before. No on-demand news, music, entertainment, chatting, reservations, e-commerce and all the others.

    Generally, larger networks are more valuable than smaller networks: if you are the only person in the world who has a telephone, that phone is not worth much. If there are 2 people with phones, you can at least call each other. With 3 people, 3 different conversations can be had. With 4, it’s 6. With 100, it’s 100*99/2 = 4950 possible conversations, not counting multi-party conference calls. This quadratic growth of value with the size of the network applies to all networks, according to Metacalfe’s Law.

    But in this post, I want to look at another dimension of networks that impacts their values, and that is whether the network is “open” or “closed”. There are lots of details one could consider, but for our purposes here, we define a spectrum with two extremes, and lots of gray in the middle:

    Fully open Somewhere in between Entirely closed
    Anybody can connect to the network and do what they like, nobody's permission is required. Who may connect, and what they may do on the network, is closely controlled by the network proprietor.

    There can be all sorts of network proprietors, but for simplicity in this post, assume it’s a single entity, like Meta.

    Here are some examples:

    Fully open Somewhere in between Entirely closed
    The public road system. Roads on a private golf course.
    Buyers and sellers using cash. Buyers and sellers using Visa. Internal company accounting system.
    The world-wide web. Facebook. Twitter. The old AOL walled garden.

    If you had two networks that are otherwise identical in size, structure and function, except that one is open and the other one is closed, which of those two is more valuable?

    Valuable to whom?

    Fully open Somewhere in between Entirely closed
    Valuable to:
    • Platform proprietor: no, does not exist
    • Network users: yes
    		 Valuable to:
    • Platform proprietor: yes
    • Network users: yes

    It’s clear that if both networks produce the same amount of total value, the open network is more valuable to its users (such as individuals and organizations), for the simple reason that there is no network proprietor who needs to get paid! The value entirely accrues to the network participants.

    But there’s more to it: Cory Doctorow recently coined the term enshittification to describe the inevitable march of platform/network proprietors, over time, to siphon off an ever-larger percentage of value generated by their network, to the detriment of its users. So the older a closed network, the less value it provides to its users. (Facebook users experience this every day: ever more ads, ever less genuine human engagement. While, for its business users, ad prices go up.) In an open network, on the other hand, the value that accrues to the users does not deteriorate over time.

    And finally: could AOL, the online service, ever have provided the same value as the open web? Of course absolutely not! Open networks allow many more technologists and entrepreneurs to innovate in a gazillion different ways that would never be possible in a closed network. As closed networks mature, not only do they enshittify, but they also further and further discourage innovation by third parties, while the opposite is true for open networks.

    Which brings us to the Fediverse. Which is more valuable today: the decentralized, open social network called the Fediverse (with its thousands of independently operated Mastodon, and other instances), or the poster closed social network, Facebook?

    Clearly, Facebook. That’s because by all counts, Facebook today has order-of-magnitude about 1000 times the number of users of the Fediverse. Same for Twitter, which has maybe 100 times the number of users of the Fediverse.

    But the network effect is the only thing the closed social platforms have going for themselves. All other parts of the value proposition favors the open social network alternative. Think of this:

    • The Fediverse extracts far less / no value: no annoying ads, no user manipulation favoring the business model of the network proprietor.
    • More functionality: it’s one interoperable social network with apps that emulate Twitter, Facebook, Medium, Reddit, Goodreads, and many others! In the same network.
    • It’s entirely open for innovation, and innovators are building furiously as we speak. By its nature, it’s permanently locked open for innovation, and there is no danger of ever getting cut off from an API, facing sudden connection charges or drawing the wrath of a gazillionaire.

    So by the time the Fediverse has sufficient numbers of users, it’s game over for proprietary social networks. This is true for both user categories in social networks: individuals and businesses. (I assume here that businesses and the Fediverse will find a way of accommodating each other, mainly by businesses behaving respectfully. If not, there simply will be no businesses in the Fediverse.) Individuals will get more value from the open network, and businesses will be far more profitable because there is no network operator to pay and many products and services pop up all the time that won’t in the closed network.

    Note that the critical “sufficient number of users” can likely be substantially smaller than the user populations of those closed networks today, because all value accrues to users and it’s not diminished by value extraction from a network proprietor. For many of my own use cases, in many niches the Fediverse has critical mass today already.

    Can the user advantage be overcome across the board? We will have to see. But if we add up just numbers of active users of organizations that have publicly announced Fediverse plans as of the date that I’m writing this, or even have products already in the market – Flipboard, Medium, Mozilla, Tumblr, Wordpress and more – we’re already in the high 100’s of millions.

    Those numbers look awfully close to the user numbers necessary to overcome Metcalfe’s Law.

    tldr; The time to take the Fediverse seriously, for individuals and businesses, is now. The value of the Fediverse for everybody is much higher than the value of any closed, proprietary social network – other than the proprietary social network companies themselves. And we won’t cry for them very much.

    Note: FediForum is next week, where we’ll discuss this.

  • 2023-03-10

    Meta’s decentralized social plans confirmed. Is Embrace-Extend-Extinguish of the Fediverse next?

    Casey Newton at Platformer reports he has e-mail confirmation from Meta that:

    [Meta is] exploring a standalone decentralized social network for sharing text updates. We believe there’s an opportunity for a separate space where creators and public figures can share timely updates about their interests (Source).

    Their new app is codenamed P92, and according to a separate report by Moneycontrol:

    … will support ActivityPub, the decentralised social networking protocol powering Twitter rival Mastodon and other federated apps (Source).

    It will also:

    be Instagram-branded and will allow users to register/login to the app through their Instagram credentials.

    First, the good news:

    This is a huge validation of the decentralized social network known as the Fediverse, built around a set of internet protocol standards that include ActivityPub, ActivityStreams, WebFinger as well as a set of commonly implemented unofficial extensions. The Fediverse has been around for some years, but recently came to more widespread prominence through its leading implementation, Mastodon, as the leading alternative of increasingly erratic (and increasingly many other things, but I digress…) Twitter.

    That’s because only when alternatives are actually beginning to look like they might become serious threats to incumbents – and Meta is the market-leading incumbent in social media by far – do incumbents start paying attention and then connect to them. Or, as it may be the case here, simply leak that they might be connecting in the future but never actually will. We don’t know which of those will turn out to be true, but it doesn’t matter: both validate the Fediverse as a serious competitor to Meta.

    This is on the heels of recent Fediverse adoption by companies such as Mozilla, Medium, CloudFlare and Flipboard. Apple now has Mastodon content previews in iMessage. Even Microsoft has been spotted in the Fediverse a few days ago.

    But:

    I have some Brooklyn Bridges for sale. You get a Brooklyn Bridge for free if you believe that a company like Meta would connect to the Fediverse, and be a perfect citizen the way the Fediverse expects you to be today. Including:

    • No ads;
    • No tracking;
    • No algorithms that favor business outcomes for Meta over your wellbeing;
    • Respect for different cultures, minorities, non-mainstream behavior etc.;
    • A rich toolset for filtering and blocking according what you decide you want to filter and block, not Meta;
    • The ability to move from one host to another without having to build your network from scratch;
    • The ability to pick who is your system administrator and moderator, from models that are essential centrally managed to full-fledged self-managed, user-owned cooperatives;
    • The ability, and encouragement, to innovate with new apps;
    • and so forth.

    Instead, based on the history of technology, the chances are overwhelming that such an app would be used by Meta with an embrace, extend and extinguish strategy, at the end of which the Fediverse would either have become irrelevant or effectively been taken over by Meta. So the much-heralded alternative to Twitter would become … Meta? I sure hope not.

    If you think that is unlikely, read up on some of the historical examples listed on that Wikipedia page. Merely being based on open standards and having a million+ strong user community does not protect you at all. Instead, I would say the attack happens every single time a network dominated by an incumbent (here: social media) is threatened by a more open network. And it succeed, at least partially, more often than not. Here it is Meta’s $100b+ business that’s under strategic threat, of course they will protect it and use any means they can think of to do so.

    It does not help that the Fediverse today is chronically underfunded and has corresponding difficulty to compete at the same speed as somebody like Meta can. Actually, “unfunded” is a better term because the amounts are so small. There are many unpaid contributions, the Fediverse largely being open source and all, but I’d be surprised if more than $10m per year are spent in total on the entire Fediverse today, likely it’s far less. If Meta can burn more than $10b – that’s one entire annual fediverse spend every 8 hours! – on a very doubtful Metaverse project, they surely could find the same amount of money to protect their core business.

    And that’s just one of the many issues we need to solve to protect, and grow, the beautiful thing we currently have with the Fediverse.

    So what shall we do about all this?

    (I welcome your comments – in the Fediverse! Find me at @j12t@j12t.social.)

    (Also, I’m co-organizing FediForum, an online unconference at the end of March, where we will surely discuss this and other issues. And celebrate the Fediverse, because there is much to celebrate! Join us?)

  • 2023-01-29

    What if Apple’s headset is a smashing success?

    Signs are pointing that Apple will announce its first headset in the next few months. This would be a major new product for Apple – and the industry beyond –, but there is very little excitement in the air.

    We can blame Meta for that. After buying Oculus, iterating over the product for almost 9 years since, and reportedly spending more than $10 billion a year on it, their VR products remains a distinct Meh. I bought a Quest 2 myself, and while it definitely has some interesting features (I climbed Mt Everest, in VR!), it mostly sits on the shelf, gathering dust.

    So the industry consensus is that Apple’s won’t amount to much either. If Meta couldn’t find compelling use cases, the thinking goes, Apple won’t either, because there aren’t any! (Other than some limited forms of gaming and some niche enterprise ones.)

    I think this line of thinking would be a mistake.

    My argument: Apple understands their customers and works down their use cases better than anybody. If Apple works on a new product category for many years – and signs are that they have – and then finally decides that the product is ready, chances are, it is. Their track record on new products is largely unblemished since the return of Jobs about 25 years ago:

    • fruity fun design for a computer (iMac) – success
    • digital music player (iPod) – smashing success
    • smartphone (iPhone) – so successful it killed and reinvented an entire industry
    • table (iPad) – success
    • watch (iWatch) – success
    • … and many smaller products, like headsets, voice assistance, Keynote etc.

    Looking for a major dud in those 25 years, I can’t really find one. (Sure, some smaller things like the 25-year anniversary Mac – but that was always a gimmick, not a serious product line.)

    It appears that based on their history, betting against Apple’s headset is not a smart move. Even if we can’t imagine why an Apple headset would be compelling before we see it: we non-Apple people didn’t predict iPhone either, but once we saw it, it was “immediately” obvious.

    So let’s turn this around. What about we instead assume the headset will be a major success? Then what?

    I believe this would transform the entire technology industry profoundly. For historical analogies, I would have to go back all the way to the early 80’s when graphical user interfaces first became widely used – coincidentally (or not) an Apple accomplishment: they represented a fundamentally different way of interacting with computers than the text terminals that came before them. Xerox Parc gave that demo to many people. Nobody saw the potential and went with it, just Apple did. And they pulled a product together that caused the entire industry to transform. Terminals are still in use, but only by very few people for very specific tasks (like system administrators).

    What if AR/VR interfaces swept the world as the GUI swept the PC?

    I believe they can, if somebody relentlessly focuses on uses cases and really makes them work. I built my first 3D prototype in VRML in 1997. It was compelling back then and it would be today. Those uses can be found, I’m quite certain.

    Based on everything we’ve seen, it’s clear that Meta won’t find them. Hanging out with your friends who don’t look like your friends in some 3D universe is just not it. But if anybody can do it, it’s Apple.

    So I’m very much looking forward to seeing what they came up with, and I think you should be, too.

  • 2023-01-24

    Activity Streams graphical model

    All you need is a gazillionaire doing strange things to some internet platform, and all of a sudden decentralized social media soars in adoption. So lots of people are suddenly seriously looking at how to contribute, myself included.

    Core to this is the ActivityPub standard, and real-world implementations that mix it with additional independently defined protocols, such as what Mastodon does.

    None of them are particularly easy to understand. So I did a bit of drawing just to make it clearer (for myself) what kind of data can be shipped around in the Fediverse. To be clear, this is only a small part of the overall stack, but an important one.

    Here are some diagrams. They are essentially inheritance diagrams that show what kinds of activities there are, and actors, etc. Posted here in case they are useful for others, too.

    And here’s how to interpret my homegrown graphical notation. (I made it up for my dissertation eons ago, and used it ever since. It has certain advantages over, say, UML or traditional ERA diagram styles. IMHO :-))