Modern XMPP project discussion - 2022-01-17

The IETF used to use XMPP for their online chats during meetings. Last year they decided to explore alternatives (including Matrix). I believe the post author confused this with standardizing XMPP alternatives, but this never anything to do with standards. In fact they ended up choosing Zulip, which isn't a standard in any way.

And that first hn comment is just hn as usual. Mixing up protocol and implementations, and ranting on serialisation format which honestly who cares

HN people care ;)

^^

It was hard convincing my family to use Snikket, once they learned it used XML

seriously?

> It was hard convincing my family to use Snikket, once they learned it used XML 😂😂

The web usea XML

Ah sorry wrong example, we do want to kill the web..

also it uses SGML, not XML

uses*

You are all wrong, HTML is totally not like XML in any way!!!

Also, nobody uses HTML anymore, it's all client-side rendered JSON

jonas’, obviously not seriously, sorry :P

pfew, thanks

The web usea XML

Ah sorry wrong example, we do want to kill the web..

uses*

-ELOOP?

Tbh I'm slightly annoyed that each time we skip the false debate on protocol/implementations and having to implement everything etc. People seem to be really lost about this, I'm wondering if it's legit lost or trolling lost

Istr matrix was also built on "not having to wander through lots of disparate documents" (because you do have to implement everything to "implement the spec"), how true is this?

Not 100% true, and their "single spec" is... very long :)

Sure, it might be in the end.. Plus nobody actually implements everything. "Everything" not even being written down in the spec

So trolling or legit? :p

Yeah, spec aside, it is absolutely the case that not every Matrix client implements the same stuff

Even Element across different platforms can differ, as things get rolled out to one platform before others

But if you don't implement everything *and* keep up with the constant stream of changes then you're not a Matrix client and you don't count!

Or in other words: There is only one implementation.

Modernxmpp is exactly tackling this false debate IMO and maybe it should address it directly

Maybe it already does and people still don't realise, dunno

I'll be honest that the "Modern XMPP" project is nowhere near what I envisioned when I first introduced it at FOSDEM 5-6 (!) years ago. I think it's working great as a place to record knowledge that is otherwise unrecorded or fragmented. However to be what I had hoped, it would need serious support from client developers, and I just don't see that anywhere.

and that's partly why I prioritize Snikket, because working on an implementation allows direct change, instead of writing documentation and hoping that all (or at least some) people will follow it... eventually

I'm not so surprised people do things differently. Every implementation has different goals, and even when having similar concepts these goals will reflect in how these concepts are implemented. I think Snikket is the closest you'll get to having something that has the same behaviour everywhere (which is still not a given).

Indeed

Maybe client developers could voice their opinion here in this regard and how they would like to proceed?

They mostly don't

Well, I should be more clear

It's not that they are against the initiative. But just look at any project's bug tracker. They have other priorities :)

It's the usual story of open-source

"do it yourself if you care for it to be done"

So even if they agree in principle, even just changing terminology can be a big task in itself

(or throw some adequate and accepted compensation)

Snikket isn't even consistent on terminology (yet), but I'm working on it

So indeed, unless we can accompany Modern XMPP with contributions and resources for the involved projects, it will never be more than a loose documentation project I think

Which is valuable to have, I'm just saying it's less than my original ambition :)

But my original ambition didn't foresee Snikket

To be honest I think Snikket is a kind-of closed solution as well. It suits specific groups of people. But nothing generic.

I claim there is no generic solution

there can be a generic protocol, XMPP is, but no generic client solution.

Ok, I must specify: With 'generic' I mean something for the vast majority of users.

Like WhatsApp, Signal, Threema,...

The reason being?

I cannot just tell someone "Install Snikket and chat with me"

you can. by sending an invite.

I'd have to setup a Snikket instance first.

Someone has to

I could as well just setup my own ejabberd and create accounts there.

I would happily race anyone onboarding a friend to XMPP going the Snikket or ejabberd/Prosody route :)

You could sink a whole day setting up a good XMPP service from scratch

So far I just say "Install Quicksy and chat with me". That works fine as long as Android is involved.

MattJ: Maybe, but thats not the point.

It kind of is

But sure, if someone wants to use Quicky's service, that's totally fine. That's what XMPP allows, after all.

I disagree. *I* am tech-savvy. I could do Snikket or ejabberd. Whatever my preference would be. But the common user wouldn't do either.

Of course they wouldn't

But you don't need tech skills either, there are plenty of non-technical people with their own Snikket instances via the hosting service

It takes a few minutes, and then you can send invites or chat to people using Quicksy, or whatever

Yes, people can do that, but I think the majority expects another workflow in this regard.

Well, we have to shift their expectations :)

One group of people at a time

And that exactly is the big problem of most developers. They think the users need to change and adopt to the developers expectations. Not the other way round.

I agree

Which is exactly why I put so much effort into making Snikket onboarding absolutely trivial

Just *one* person has to set up the instance, and that's easy enough for at least one person in most groups.

Don't get me wrong, I think Snikket is a very good idea, but it targets people who are willing to take responsibility for managing a group of people.

If you prefer to promote a service like Quicksy, go for it. I won't judge :)

I have personal opinions that the XMPP public server model and the Quicksy model are both the wrong approach. Snikket's model is the closest to ideal that I have come up with so far.

But as long as it's all XMPP, who cares which one "wins" (we don't even need a winner... that's the point)

The thing is that it seldom grows beyond this group of people. You have this one person and his peers on the Snikket instance. One of those peers will usually not tell a third person to "install Snikket". He would either setup his own instance first or tell the one person to send this third person an invite.

He would either *have to* setup his own...

Ordinary users will be able to invite new users to instances, in the future

If your concern is that the admin is the bottleneck

Hm, then the admin will have to agree that people invite further people who he might not know.

Yes, that's why it's not allowed right now, because there aren't sufficient controls for the instance admin

There will always be controls. My goal is not to replicate a centralized service on top of XMPP (e.g. Quicksy). There's really not much point in that, we might as well stick with existing centralized services (who arguably do it better).

Meanwhile Quicksy has to fight with network effect, while Snikket mostly does not

> might as well stick with existing centralized services (who arguably do it better) which ones? usually they don't federate, so they're inaccessible

But from what I can see you're saying that this is the way forward, because of user expectations that cannot be changed

I assume you reckon a small minority of people will run federated services alongside a giant service with millions of users

or, explain :)

We had such a service once before - Google Talk

MattJ: We might as well have the problem, that someone sets up a Snikket instance (because it's so damn easy), then invites some users who invite further users and then there are a growing number of users on this instance. But the one person who maintains the instance is maybe not that tech-savvy (it's so easy, that's your point isn't it) and doesn't do updates. Then he finally re-installs his system without taking care. Well... what do those users say then?

That's why we will have controls and limits in place

I think someone who sets up a server needs to have some kind of responsibility and take care as well.

The current plan is for users invited by non-admins to be unable to invite other users by default

Then we have the problem I mentioned in the first place. I think the network effect is the most important part of XMPP. And we only get that with features I guess. And working clients for all common OS.

Seems we have gone in a circle :)

Thats what happens with XMPP for years I'm afraid. :(

Maybe everything goes in circles...

Luckily Snikket supports those

:-P

This is now the canonical reason for calling them that

> working clients for all common OS. the only one missing is iOS now

Depends what you mean by "working" and "client", I guess

I assume the lines are drawn to exclude Snikket iOS, which obviously I disagree with but I see why others do not :)

I have a friend who registered her account on 404.city and installed Siskin. She doesn't receive notifications for group chat and all messages just read "You received a message encrypted with OMEMO but your client doesn't support OMEMO. " I have to send her screenshots from our conversation in the group chat

as far as I know Snikket iOS is based on Siskin IM?

Yes

It requires additional extensions on the server side for reliable notifications

and these aren't under the XSF yet, they are published on Tigase's website

Which is one reason ejabberd doesn't have them yet

maybe she could fix the notification problems by migrating to a Tigase server like sure.im, but I suspect the OMEMO situation will not improve...

Why are these extensions not under the XSF yet?

Mostly because nobody submitted them

At the time they were introduced to the community, a number of folk spoke against them, because they use JSON embedded in XML

So the Tigase team didn't really pursue publishing them through the XSF due to lack of interest

> a number of folk spoke against them, because they use JSON embedded in XML I think I have to agree, that sounds terrible. Why would that be necessary?

(I'm not trying to speak for the actions of the Tigase team here, just my assumption is that had there been more positive reaction, they would have submitted them)

Because the Apple push API has a strict size limit

The data gets encrypted on top of that before it is passed to Apple

It's more compact to pick a few fixed fields and encode them as compact JSON

i.e. there is no extensibility

Is there so much information that needs to be submitted that it exceeds the size limit?

Which is part of the problem

I forget the limit exactly, but "4KB" rings a bell in my mind. But a JID maximum length is already >2KB

and then there's any message contents, OMEMO info, Jingle call info, group JID, etc.

but the encrypted client <--> apple API should not be problem of the parts XMPP cares about namely c2s and s2s, right?

Well that sounds at least like a valid reason but it could be discussed and then decided for either a better solution or then accept the current solution. Not doing anything because there is no perfect solution is the worst.

This data is encrypted XMPP server-->Apple-->client

Basically the server receives an incoming stanza addressed to the client, and the server has to pack this in some form and send it to the client via Apple's API

and the details of the packing are the main problem

In an ideal world the client should be able to tell the server what info it needs from an incoming stanza

"If it's a Jingle call, extract this info, pack it like this, encrypt it, and send it to me as a push notification"

But this starts to get pretty complex

Greenspun's tenth rule?

In the current Tigase approach, the information to extract is hard-coded in their XEP

is it a XEP if it isn't published by the XSF?

Which is fine for a single implementation, or other very similar implementations, but if someone wanted to e.g. add in "this is a sticker" flag in there, suddenly it's problematic

XMPP External Proposal

nice save

(though "it's proprietary magic" is one of the criticisms I hear of Snikket (iOS))

If someone could figure out the spec we could switch pretty easily to something standardized at the XSF

But if that person has to be me, it's going to have to wait

So it’s a similar issue to EXI?

A client wants to share a list of what it supports/wants to receive notifications for with the server?

I don't think its strictly similar

but it could be

I wonder if we could get away with a list of simplified XPath selectors

as opposed to the word XSLT someone mentioned somewhere, which would be horrible

I don't understand what kind of extensions are needed for the c2s part? How the server talks to the Apple API would never be part of an XEP.

zaik, the client needs to tell the server what to send.

zaik, the data it sends to the push server must be part of the XEP

which parts of the stanza it needs *in the push notification itself* in order to operate

zaik, it would be, since despite Apple being the relay, it receives the payload the server sends it.

or some way to determine what data to send (ultimate flexibility)

:D

MattJ, have it be a WebASM program! ✏

You meant Lua script, right?

no, webasm

I sure would love having clients upload wasm scripts to their server for processing. :D

though I bet it can be transpiled in that direction, too

Link Mauve, let me tell you about that pubsub option which allows clients to upload an XSLT to generate the notification <message/> ✏

I know about it.

I'm sure you can build a WebASM interpreter in XSLT

I even implemented it in PSĜS IIRC.

jonas’, sure, it is turing-complete.

yeah, I know, even though I haven't implemented a WHILE in it yet

I should do that

I also have rust macros in my list for that

(macro_rules!, not proc macros, that would be boring)

:D

I also think I might have accidental turing completeness in my metric relay tool, I need to evaluate that

Oh no.

> She doesn't receive notifications for group chat and all messages just read "You received a message encrypted with OMEMO but your client doesn't support OMEMO. " > I have to send her screenshots from our conversation in the group chat Oh so that's what happens? :o is it all just about notifications? I also have a friend in this case and it's awful experience

she doesn't get notifications and can't decrypt the messages.

(in group chat)

pep., it's not clear to me what would cause a decryption failure like this, but it's the kind of thing I hope to get to the bottom of with things like mod_debug_omemo (eventually)

Hence my question to Daniel about Conversations' logic for this case yesterday

From what he says, sending a message to the group ought to be enough

some indication on the client side about what kind of error is occuring would also be helpful...

Siskin does, iirc

If you get the error icon next to the message you can show more details

If you don't get any error, that also narrows things down :)

I mean, I haven't used vanilla Siskin for a while, but maybe it's as simple as that user having OMEMO disabled for the group

https://upload.wiuwiu.de/share.php/6a98048d-3ec7-4417-9690-6e07b82b2e81/a414b065-d1eb-4ed1-be97-414dcf71d540_1.jpg

So it looks like messages from B can be decrypted, but not messages from S or N?

B is herself :)

Ah

Which is encrypted. So yes, very strange.

and no indication what has gone wrong :(

S and N are using Conversations?

My 90% guess is that the message was simply not encrypted for B's device

But this is up to the sender's client

That's the problem with debugging things like this - there are so many variables, and they are everywhere

e.g. maybe the sender couldn't fetch the keys for some reason

> S and N are using Conversations? yes, i am S and on my side all of her keys are 'on'

Which is why I'm trying to gradually surface this kind of debug info in Snikket, so it's accessible when the need arises

If you have the ability to capture the XML of a message that you send, and one that you receive, it would confirm, at least partially, where the problem lies

thanks, i can do that when i get home

The sooner we can knock out issues like this, the better for everyone

MattJ, I now have the XML capture of a message I sent and one I received from by friend B:

https://upload.wiuwiu.de/share.php/8dd70367-1ace-40f1-8d91-ad437714c158/debug.xml

The first is sent by you, the second is sent by them?

(just to confirm)

It looks okay to me, both messages include a key for the other

> The first is sent by you, the second is sent by them? Yes. ✏

> It looks okay to me Hm 😕️

It's annoying that these problems always happen on user devices, not developer devices :)

of course :D

I would like to have the option to send someone a link to a website where they are able to take part in a MUC conversation _without_ installing an app and _without_ creating a JID.

Private MUCs or public?

In my special case it would be a public room.

There used to be a thing for that, it was at speeqe.com or such (I'm pretty sure the domain is long expired, so I wouldn't recommend visiting the site now)

It had various problems with abuse, as you can imagine. But I suspect we have better tooling to deal with things like that these days (though not eliminate it entirely)

Basing it on invitation links would indeed be interesting

Of course one should not be able to join anonymously just like that. Just some auth-token that is valid for a specific timeframe given as URL parameter and generated from a client. Maybe even including the nickname and/or the user who created the auth token.

This way people could join with a simple click on the link. Possibly some of them will later install an app for convenience. The link is just to get them hooked.

Isnt that kind of what you can do qith your snikket server and the application? From wnat i have heard at least

> _without_ installing an app and _without_ creating a JID.

Snikket needs a running and maintained Snikked instance.

though I supposed by "without creating a JID" zak meant "without creating a persistently existing or managed JID". you cannot participate in a MUC without any JID :)

Luckily running and maintained Snikket instances are easy to obtain :)

jonas’, of course. I meant "without the need for the user to create a JID"

But yeah, I don't think Snikket is entirely for the requested use-case

zak: yeah of course. But the idea is there

The thing for me is, if I start setting up a Snikket instance and get people onboard there, I would have to keep the instance running for... ever? I don't want to bind myself to this responsibility.

zak, thanks to account import/export or even the possibility of transferring the instance to a hosted service provider, not necessarily

Why do you have to keep it forever? You can just set the expectations straight. Not always up/ may close down

msavoritias, that's terrible

Same as just about any internet service at this point

jonas’, how exactly is the workflow for the users?

I promote xmpp as yoy have to pay for your server.

But with better ability to migrate to (XMPP-based) alternatives

jonas’: why

zak, if you transfer to a hosted service provider: users are unaffected, just export a backup of your snikket and re-import it at the hosted service provider.

if you do account import/export, every user has to do things and it is very disruptive to their rosters; the story around moved (xep-0283) isn't fully sorted out yet

jonas’, won't they get a new JID?

zak, no, you can carry the domain in the former case

if the hosting provider offers bring-your-own-domain (which the official one does)

jonas’, I don't want to host somewhere else. I don't want to host at all.

well, a chat account consumes resources, and someone needs to pay for that, one way or the other.

That's fine, I just want to be able to let people use one of the already existing servers.

Such as blabber.im

(sorry, couldn't resist)

Exactly. And exactly the same will happen if people start setting up Snikket instances because it's so easy.

I'm not convinced ✏

running a public server has very different incentives and energy/reward trade-offs compared to a private family server

sure. And there is nothing against setting up a (Snikket- or whatever) Family server. ✏

I am thinking about use-cases and workflows for WhatsApp-, Signal, etc.- users.

zak, what do they do when WhatsApp or Signal closes down?

This wouldn’t be the first time a once-huge service shuts down without any migration path.

Link Mauve, if one of both happens everyone currently in this room gets a bottle of wine from me. 😉️

Well just ask google or msn

As I said...

Disclaimer: Renaming doesn't count.

msavoritias, well, Google has some compatibility between their different chat services, and MSN got renamed to Skype, it’s Skype which stopped existing actually.

Skype -> Skype Business -> MS-Teams

Google did shut down Google+ for instance, with no fallback possible for users.

Still a migration path

My point is that its better to aim for a lot of servers people van migrate to and make that migration easier. Instead of megaservers that may or may not be here tomorrow.

+1

Before people can migrate, they need to onboard first.

thats true. personally i would love for the snikket model of invite to be everywhere.

then i dont have to tell peofle to use passwords and go to obscure sites

What if the invite-feature could be integrated within ejabberd and/or prosody? But I see that MattJ probably wants this to be "Snikket".

Isnt it already a XEP? Or am i wrong?

As far as i know anybody can implement it

It's already a XEP

It's supported in Prosody, Conversations, Siskin, yaxim, and probably others

An ejabberd implementation at some point is hopeful :)

How can you do it in Conversations actually? Do you just open the invite?

Yes

Well then I just need to contact my hoster to enable that feature soon. 😀️

Conversations supports both xmpp: format invites and https://conversations.im/i/

Ah wait, I think this is different? The account creation is separate from the invite feature, isn't it?

and then there is the contact-discovering issue... all integrated in Snikket only.

Implemented practically everywhere, integrated... in Snikket

I cant wait for conversations 3. Maybe some defaults change then

msavoritias, which defaults in particular?

One of them being the invite to be more prominent. Maybe contact discovery too. Also new omemo and mix for groups would be nice among others

I wrote a short paragraph on the OSM wiki on how to join the IRC channels from an XMPP client here: https://wiki.openstreetmap.org/wiki/IRC If I click open the MUC link in Conversations, it says 'invalid XMPP address'. Anyone know how to fix the URL?

zaik, most likely you forgot to encode the # and % for URIs.

Just like in http: URIs you have to escape e.g. a space character into %20 (I’m sure you’ve experienced that already), in xmpp: URIs it’s the same.

%23 and %25 for these two characters respectively.

zaik, also you made a typo, the text says #osm but the link links to #osm-at.

if i use %23 and %25 it doesn't work in gajim anymore... :( > text says #osm but the link links to #osm-at that's true I should fix that

Making XMPP an alternative to IRC would be another approach.

approach to what?

Well xmpp is kind of an alterative. At least from what i read it was intended to be. Poezio goes that way. But i dont think it makes much sense

Oh huh, yah, hadn't noticed this before but Gajim appears to be broken and isn't doing URL decoding

lovetox: ^

zaik, the correct way is to decode the URI encoding, Gajim is at fault here.

approach to get XMPP to more people

thanks, I fixed it now and it works with Conversations :)

Also we do have a French OSM room at xmpp:osm-fr@chat.jabberfr.org?join, you get encouraged to join it whenever you do a modification on any part of France at https://osm.org

Other local OSM communities could do the same instead of relying on IRC. ✏

Currently there are France, Germany, Hamburg and Berlin-Brandenburg OSM communities on XMPP: https://community.osm.be/

I would start an local OSM MUC, but I don't know anybody who would join :(

Those MUCs need to be advertised on the webpage as well of course.

not sure if a MUC with a single occupant qualifies for the community index...

i think there needs to be an community not just the intention of one

There are "Community" links with IRC-Chats on almost every bigger software project page. If only a third would change these for XMPP-MUCs at least more people would know about XMPP.

as I pointed out elsewhere… I don't see much value in changing those into XMPP

IRC is a good baseline

if they don't like IRC anymore, then sure, that's a different question.

but a community happy with IRC changing to something else can only lose

because both XMPP and Matrix bridge well to IRC

I don't like IRC. It's like Perl, but without Features.

better IRC than Matrix

Irc is nice for what it is. Ephemeral, open rooms

Sure. But the Matrix train is already relling.

s/Ephemeral/Ephemeral or long-lived/

jonas’: definetily. I am not creating a matrix account for projects

I am not really against IRC, but I think XMPP could find users there. Just for promoting XMPP more.