This assumes that everyone implements the same set of APIs that work in the same way.
More likely, the browser will implement some that make sense there, some browsers will implement more than others, Cloudflare workers will implement a different set, AWS Lambda will implement a different set or have some that don't work the same way... and now you need to write your WASM code to deal with these differing implementations.
Unless the API layer is, essentially, a Linux OS or maybe POSIX(?) for Docker, which I doubt it would be as that's a completely different level of abstraction to WASM, I don't have a lot of faith in this being a utopian ideal common API, given that as an industry we've so far failed almost every opportunity to make those common APIs.
Good point! This is the hard work that people are undertaking right now.
Things are going to change a little bit with the introduction of Preview3 (the flagship feature there is async without function coloring), but you can look at the core interfaces:
This is what people are building on, in the upstream, and in the bytecode alliance
You're absolutely right about embeddings being varied, but the standard existing enforces the expectations around a core set of these, and then the carving out of embeddings to support different use cases is a welcome and intended consequence.
WASI started as closer to POSIX now, but there is a spectacular opportunity to not repeat some mistakes of the past, so some of those opportunities are taken where they make sense/won't cause too much disruption to people building in support.
It isn't the fault of the group that suggests and standardizes protocols, it is everyone thinking they are smarter and they can do it better is the problem.
> the flagship feature there is async without function coloring
Correct me if I’m wrong, but that’s only possible if you separate runtime threads from OS threads, which sounds straightforward but introduces problems relating to stack-lifetimes in continuations so it introduces demands on the compiler and/or significant runtime memory overhead - which kinda defeats the point of trying to avoid blocking OS threads in the first place.
I’m not belittling the achievement there - I’m just saying (again, correct me if I’m wrong) there’s a use-case for function-colouring in high-thread, high-memory applications.
…but if WASI is simply adding more options without taking anything away then my point above is moot :)
Further to this, my (very basic) understanding is that the actual threading implementation will be left up to the integrator, so some implementations may not actually implement any concurrency (a little like the Python GIL in a way), while others may implement real concurrency, therefore meaning that subtle threading bugs could be introduced that wouldn't be seen until you run in other environments.
> Correct me if I’m wrong, but that’s only possible if you separate runtime threads from OS threads, which sounds straightforward but introduces problems relating to stack-lifetimes in continuations so it introduces demands on the compiler and/or significant runtime memory overhead - which kinda defeats the point of trying to avoid blocking OS threads in the first place.
Correct -- note that the async implementation does not address parallelism (i.e. threading) -- it's a language +/- runtime level distinction.
The overhead is already in the languages that choose to support -- tokio in rust, asyncio in python, etc etc. For those that don't want to opt in, they can keep to synchronous functions + threads (once WASI threads are reimagined, working and stable!)
You can actually solve this problem with both multiple stacks and a continuation based approach, with different tradeoffs.
> I’m not belittling the achievement there - I’m just saying (again, correct me if I’m wrong) there’s a use-case for function-colouring in high-thread, high-memory applications.
>
> …but if WASI is simply adding more options without taking anything away then my point above is moot :)
Didn't take it as such! The ability to avoid function coloring does not block the implementations of high-threads/high-memory applications, once an approach to threading is fully reconsidered. And adding more options while keeping existing workflows in place is definitely the goal (and probably the only reasonable path to non-trivial adoption...).
How to do it is quite involved, but there are really smart people thinking very hard about it and trying to find a cross-language optimal approach. For example, see the explainer for Async:
There are many corners (and much follow up discussion), but it's shaping up to be a pretty good interface, and widely implementable for many languages (Rust and JS efforts are underway, more will come with time and effort!).
True. Its probably worth creating a validation suite for wasi which can check that any given implementation implements all the functions correctly & consistently. Like, I'm imagining a wasm bundle which calls all the APIs it expects in every different configuration and outputs a scorecard showing what works properly and what doesn't.
I suspect you're right - unless people are careful, it'll be a jungle out there. Just like javascript is at the moment.
I understand your point but sadly I think it's too idealistic (not that we shouldn't strive for these goals). We already have those sorts of tests for browsers, and browser compatibility is still a problem. We have acceptance tests for other areas, like Android's CTS tests, but there are still incompatibilities.
That also assumes that everyone involved wants compatibility, and that's unlikely. Imagine a world where every WASM implementation is identical. If one implementation decides to change something to implement an improvement to differentiate themselves in the market, they'll likely win marketshare from the others.
Most companies implementing WASM will tend to want to a) control the spec in their own favour, and b) gain advantages over other implementations. And this is why we can't have nice things.
> I understand your point but sadly I think it's too idealistic (not that we shouldn't strive for these goals). We already have those sorts of tests for browsers, and browser compatibility is still a problem. We have acceptance tests for other areas, like Android's CTS tests, but there are still incompatibilities.
>
I think the browser problem is a marketshare/market power problem, and Wasm doesn't have that problem.
Also, I'd argue that compat tests for JS engines and browsers are an overall positive thing -- at least compared to the world where there is no attempt to standardize at all.
> That also assumes that everyone involved wants compatibility, and that's unlikely. Imagine a world where every WASM implementation is identical. If one implementation decides to change something to implement an improvement to differentiate themselves in the market, they'll likely win marketshare from the others.
This is a good thing though -- as long as it happens without breaking compatibility. Users are very sensitive to changes that introduce lock-in/break standards, and the value would have to be outsized for someone to forgo having other options.
> Most companies implementing WASM will tend to want to a) control the spec in their own favour, and b) gain advantages over other implementations. And this is why we can't have nice things.
I think you can see this playing out right now in the Wasm ecosystem, and it isn't working out like you might expect. There are great benefits in building standards because of friction reduction for users -- as long as there is a "standards first" approach, people overwhelmingly pick it if functionality is close enough.
Places that make sense to differentiate are differentiated, but those that do not start to get eaten by standards.
I think organizations that are aware of this problem and attempt to address is directly like the Bytecode Alliance are also one of the only forms of bulwark against this.
> I think the browser problem is a marketshare/market power problem, and Wasm doesn't have that problem.
No, it really isn’t.
For more than the last two decades every browser bar IE looked towards compatibility and only included differences as browser-specific extensions.
And even when Microsoft eventually caved and started the Edge project to create a compatible browser, they ended up admitting defeat and pivoted to Chromium themselves.
Maybe I'm just not understanding, but I'm not sure how this precludes it being a marketshare problem -- the thing is that the marketshare leader doesn't have to worry about compatibility/being interoperable.
> And even when Microsoft eventually caved and started the Edge project to create a compatible browser, they ended up admitting defeat and pivoted to Chromium themselves.
This can be interpreted as a problem of marketshare not staying balanced. It may have shifted hands, but the imbalance is the problem -- if Chrome had to deal with making changes that would be incompatible with half the users that visit sites on Chrome, they'd be forced to think a lot more about it.
This doesn't mean they can't add value in the form of non-standardized extensions -- that's not a desirable goal because it would stifle innovation. The point is that at some point if users are on browser Y and they get a "this site only runs on browser X", they're just not going to visit that site, and developers are going to shy away from using that feature. In a world with lopsided marketshare, there's not much incentive for the company with the most marketshare to be interoperable.
IE hasn’t been the market share leader in a long time and couldn’t even retain compatibility with itself, let alone any ACID tests nor wider formalised standards.
And these days the problem is simply that the specifications are so complex and fail mode so forgiving that it’s almost impossible for two different implementations to output entirely the same results across every test suite.
Neither of these are market leader problems. The former is just Microsoft being their typical shitty selves. While the latter is a natural result of complex systems designed for broad use even by non-technical people.
> We already have those sorts of tests for browsers, and browser compatibility is still a problem. We have acceptance tests for other areas, like Android's CTS tests, but there are still incompatibilities.
Yeah - but its barely a problem today compared to a few decades go. I do a lot of work on the web, and its pretty rare these days to find my websites breaking when I test them on a different web browser. That used to be the norm.
I think essentially any time you have multiple implementations of the same API you want a validation test suite. Otherwise, implementation inconsistencies will creep in. Its not a wasm thing. Its just a normal compatibility thing.
Commonmark is a good example of what doing this right looks like. The spec is accompanied by a test suite - which in their case is a giant JSON list containing input markdown text and the expected output HTML. Its really easy to check if any given implementation is commonmark compliant by just rendering everything in the list to HTML and checking that the output matches:
> Most companies implementing WASM will tend to want to a) control the spec in their own favour, and b) gain advantages over other implementations. And this is why we can't have nice things.
Your cynicism seems miscalibrated. We have hundreds of examples of exactly this kind of successful cross-company collaboration in computing. For example, at the IETF you'll find working groups for specs like TCP, HTTP, BGP, Email, TLS and so on. The HTTP working group alone has hundreds of members, from hundreds of companies. WhatWG and the W3C do the same for browser APIs. Then there's hardware groups - who manage specs like USB, PCI / PCIe, Bluetooth, Wifi and so on. Or programming language standards groups.
Compatibility can always be better, but generally its great. We can have nice things. We do have nice things. WASM itself is an example of that. I don't see any reason to see these sort of collaborations stopping any time soon.
More likely, the browser will implement some that make sense there, some browsers will implement more than others, Cloudflare workers will implement a different set, AWS Lambda will implement a different set or have some that don't work the same way... and now you need to write your WASM code to deal with these differing implementations.
Unless the API layer is, essentially, a Linux OS or maybe POSIX(?) for Docker, which I doubt it would be as that's a completely different level of abstraction to WASM, I don't have a lot of faith in this being a utopian ideal common API, given that as an industry we've so far failed almost every opportunity to make those common APIs.