Calling wasm function pointers from JS regressions

[juj]

There are at least two code syntax regressions that occurred due to #12059 .

1. Calling a wasm function pointer from either EM_ASM, EM_JS, JS library file or external JS code (--pre-js, --post-js or external JS/<script> in HTML file) via static binding dynCall_sig(funcPtr, arg1, arg2, ...) no longer works.

E.g.

a.c

#include <stdio.h>
#include <emscripten.h>

void foo(int param1, int param2)
{
	printf("Received %d and %d\n", param1, param2);
}

typedef void (*fptr)(int, int);

EM_JS(int, callFunc2, (fptr funcPtr, int param1, int param2), {
	dynCall_vii(funcPtr, param1, param2);
});

extern void callFunc(void (*foo)(int, int), int param1, int param2);

int main()
{
	callFunc(foo, 42, 100);
	callFunc2(foo, 42, 100);

	EM_ASM({
		dynCall_vii($0, $1, $2);
	}, foo, 42, 100);
}

library_a.js

mergeInto(LibraryManager.library, {
	callFunc: function(func, param1, param2) {
		dynCall_vii(func, param1, param2);
	}
});

emcc a.c --js-library library_a.js -o a.html

It looks like the intended replacement is the several orders of magnitude slower dynamic dynCall('sig', funcPtr, [arg1, arg2, ...]); dispatch(?), but that is not good enough, and should not be used by anyone, unless they are really really in a polymorphic situation with their function pointer signature.

dynCall() is currently not available in -s MINIMAL_RUNTIME=1 builds.

2. Calling a wasm function pointer from JS library file via static binding {{{ makeDynCall('sig') }}}(funcPtr, arg1, arg2, ...); No longer works. Added test and fix for that in PR Fix regression that broke calls to makeDynCall in JS library code. #12732 .

[sbc100]

I agree with most of this. We should have done better to avoid breaking external library authors who use this API.

A few notes:

1. dynCall is just a normal library function so it can be available in MINIMAL_RUNTIME too if you declare a dependency on it.

2. dynCall can be really fast since it normally resolves directly to a table.get(). It's only slow if you end up hitting the legacy path (which can happen for example if you have i64 in your signature and you don't have WASM_BIGINT enabled). Perhaps we can figure out a better API here that is more explicit.

3. The old dynCall_sig functions are still generated in the cases where they are needed. When running in legacy mode for some reason, or when i64 is part of the signature and WASM_BIGINT is not enabled. Perhaps we can have makeDynCall use these functions directly rather than indirectly to avoid any slowdown?

[sbc100]

I think it would be great if we could find a way to completely remove the dynCallLegacy runtime function and replace it with something that is purely compile time.

[juj]

1. `dynCall` is just a normal library function so it can be available in `MINIMAL_RUNTIME` too if you declare a dependency on it.

Ah gotcha - did not look deep into that.

2\. `dynCall` _can_ be really fast since it normally resolves directly to a `table.get()`.   It's only slow if you end up hitting the legacy path (which can happen for example if you have i64 in your signature and you don't have WASM_BIGINT enabled).

That is not accurate, for several reasons:

1. WASM_BIGINT is detrimental to performance since it uses arbitrary long precision numbers, which are a lot slower than just a pair of primitive i32s. So I would not enable WASM_BIGINT in production code. Measuring performance of WASM_BIGINT is difficult in this specific area, since it depends on how much other work one needs to perform on the data.

2. The performance hit occurs for all dispatches when WASM_BIGINT is not enabled, as the implementation of dynCall resolves to

  function dynCall(sig, ptr, args) {
      // Without WASM_BIGINT support we cannot directly call function with i64 as
      // part of thier signature, so we rely the dynCall functions generated by
      // wasm-emscripten-finalize
      if (sig.indexOf('j') != -1) {
        return dynCallLegacy(sig, ptr, args);
      }
      assert(wasmTable.get(ptr), 'missing table entry in dynCall: ' + ptr);
      return wasmTable.get(ptr).apply(null, args)
    }

That is, when not using WASM_BIGINT support, all dynamic signature dispatches first do a substring search, and that is already slower than the performance of static signature dispatch.

3. Dynamic signature dispatches take in an array [param1, param2, ..., paramN] and they use .apply(null, args) to dispatch. Both of those are really bad on performance.

Benchmarking dynamic dispatch vs static dispatch, in current master branch with

#include <stdio.h>
#include <emscripten.h>

void foo(int param1, int param2)
{
}

typedef void (*fptr)(int param1, int param2);

EM_JS(int, callFunc, (fptr func, int param1, int param2), {

	// Do periodic testing, maybe a JIT only kicks in after having yielded back to main loop?
	setInterval(function() {
//		var t0 = process.hrtime();
		var t0 = performance.now();
		for(var i = 0; i < 1000000; ++i) {
			dynCall('vii', func, [param1, param2]);
			dynCall('vii', func, [param1, param2]);
			dynCall('vii', func, [param1, param2]);
			dynCall('vii', func, [param1, param2]);
			dynCall('vii', func, [param1, param2]);
			dynCall('vii', func, [param1, param2]);
			dynCall('vii', func, [param1, param2]);
			dynCall('vii', func, [param1, param2]);
		}
//		var t1 = process.hrtime();
		var t1 = performance.now();
//		console.log('8 million calls took ' + ((t1[0]-t0[0])*1e3 + (t1[1]-t0[1])/1e6) + ' msecs.');
		console.log('8 million calls took ' + (t1-t0) + ' msecs.');
	}, 1000);
});

int main()
{
	callFunc(foo, 42, 100);
}

and in Emscripten 2.0.1 with static dispatch:

#include <stdio.h>
#include <emscripten.h>

void foo(int param1, int param2)
{
}

typedef void (*fptr)(int param1, int param2);

EM_JS(int, callFunc, (fptr func, int param1, int param2), {

	// Do periodic testing, maybe a JIT only kicks in after having yielded back to main loop?
	setInterval(function() {
		//var t0 = process.hrtime();
		var t0 = performance.now();
		for(var i = 0; i < 1000000; ++i) {
			dynCall_vii(func, param1, param2);
			dynCall_vii(func, param1, param2);
			dynCall_vii(func, param1, param2);
			dynCall_vii(func, param1, param2);
			dynCall_vii(func, param1, param2);
			dynCall_vii(func, param1, param2);
			dynCall_vii(func, param1, param2);
			dynCall_vii(func, param1, param2);
		}
		//var t1 = process.hrtime();
		var t1 = performance.now();
//		console.log('8 million calls took ' + ((t1[0]-t0[0])*1e3 + (t1[1]-t0[1])/1e6) + ' msecs.');
		console.log('8 million calls took ' + (t1-t0) + ' msecs.');
	}, 1000);
});

int main()
{
	callFunc(foo, 42, 100);
}

I get the following results:

Dynamic dispatch Emscripten master:

Firefox 82:
8 million calls took 1018.2485000000015 msecs.
8 million calls took 993.6064000000006 msecs.
8 million calls took 868.5576999999976 msecs.

Node.js v12.18.1:
8 million calls took 667.679499 msecs.
8 million calls took 691.2625009999999 msecs.
8 million calls took 656.869501 msecs.

Chrome 86:
8 million calls took 783.2649999909336 msecs.
8 million calls took 782.0150000043213 msecs.
8 million calls took 755.8949999947799 msecs.

Static dispatch Emscripten 2.0.1:

Firefox 82:
8 million calls took 43.646599999998216 msecs.
8 million calls took 44.28019999999924 msecs.
8 million calls took 44.64179999999942 msecs.

Node.js v12.18.1:
8 million calls took 93.557801 msecs.
8 million calls took 94.0683 msecs.
8 million calls took 93.5231 msecs.

Chrome 86:
8 million calls took 98.53499999735504 msecs.
8 million calls took 98.02999999374151 msecs.
8 million calls took 110.5400000087684 msecs.

So function pointer calls have become ~7.23x slower in Node.js, ~23.1x times slower in Firefox, and ~7.83x slower in Chrome. This is not using dynCallLegacy or 64-bit signatures, but just plain ints.

I would recommend strongly downplaying dynamic signature dispatch, it should only be used when one really has a dynamic dispatch problem. When one has static knowledge of the signature, static dispatch should always be used. This should be the case even for "cold" code, since many applications do not need to ever do any dynamic dispatch, so can avoid codegenning that function in in the first place.

3\. The old `dynCall_sig` functions are still generated in the cases where they are needed.

That does not seem to be the case, or the "where they are needed" bit has somehow changed? See the test case in the first comment. In old Emscripten 2.0.1 and older, the fast static dynCall_sig signatures were generated, but starting from 2.0.2, they no longer are.

[sbc100]

WASM_BIGINT is detrimental to performance since it uses arbitrary long precision numbers, which are a lot slower than just a pair of primitive i32s

This part I was not aware of. @kripken are you aware of this?

One good thing is that don't currently have any dyanmic calls with i64 signatures (at least not any emscripten core). embind is one place they can arise.

Regarding your performance number, do you still see a slowdown when comparing just:

dynCall_xx(func, arg...)

to:

table.get(func)(arg..)

The work I've doing has been assuming these a roughly equivalent in terms of performance (I mean logically they are the same right?) . Hopefully change to optimize makeDynCall will reduce it to just table.get for all the cases where the signature is statically known.

[sbc100]

WASM_BIGINT is detrimental to performance since it uses arbitrary long precision numbers, which are a lot slower than just a pair of primitive i32s

Presumably callers of dynCall_vj have a single Number they want to pass are arg0 to the function (It might even be a BigInt already). I would hope that it was faster to pass this is a BigInt than to decompose it into two separate JS numbers and pass them as i32? At least I think that is what we are aiming for? @jakobkummerow?

[jakobkummerow]

I don't know what WASM_BIGINT does in Emscripten. Does it translate C++ int64_t parameters to a pair of 32-bit Wasm/JavaScript function parameters? If so, then there will likely indeed be cases where the latter is faster, but there will also be cases where the former is faster (in particular, when the 32-bit numbers don't fit into a "Smi" and have to allocate a "HeapNumber" instead, which should be no faster than allocating a 64-bit BigInt, and causes additional conversions back and forth).

Generally speaking, on the engine side, Wasm-BigInt integration being disabled means that Wasm functions with i64 parameters/results cannot be exported to or imported from JavaScript (the attempt will fail validation). Turning on Wasm-BigInt integration does not affect the performance of any functions that don't use i64 parameters/results.

So overall I'm surprised by the claim that "WASM_BIGINT is detrimental to performance", but don't know enough to outright dispute it.

[sbc100]

Sorry I should have explained WASM_BIGINT.

emscripten historically converts i64 wasm paramters to pairs of i32s. This involves a binaryen pass to re-write all such imports/exports. On the JS side we then decompose numbers into pairs of numbers before calling such functions.

When WASM_BIGINT is enable we assume the engine supports BigInt/i64 integration and we avoid the binaryen pass and the JS helper code. So I guess the question is: can/should we assume that using the BigInt/i64 integration will not be a performance regression over using pairs of i32s?

[kripken]

I have done some profiling on WASM_BIGINT, and it has been slightly faster. However, those were small microbenchmarks because I couldn't actually find anything in the emscripten benchmark suite, or elsewhere, that actually uses i64s on a fast code path. (The biggest difference I found in a very artificial one was an 18% speedup with WASM_BIGINT.)

Overall I think it's rare for i64s to make a difference at all to speed. And WASM_BIGINT lets us emit smaller code.

[juj]

The point about WASM_BIGINT was that we are now paying a performance and code size hit to cater to the machinery that enables WASM_BIGINT when NOT compiling with WASM_BIGINT support enabled. That is not acceptable performance- and code size -wise, but we can deal with that as a separate story.

Let me try to get the conversation back on track to the real problem I wanted to propose. The issue is that static signature dispatch dynCall_sig no longer works, as shown as the code in the first comment, which fails to run.

@sbc100 mentioned that it should still work. But it does not (try the code out!). He also mentioned that one should use dynamic signature dispatch dynCall() instead, to which I pointed out with the benchmark that it is not nearly the same performance. We do not want to go back to dynamic signature dispatch after #4894 took us off from it.

Is there a way that we can get static signature dispatch back with the dynCall_sig syntax to make the code in comment 0 to work?

See also the comment in #12732 (comment).

[juj]

Regarding your performance number, do you still see a slowdown when comparing just:

dynCall_xx(func, arg...)

to:

table.get(func)(arg..)

The work I've doing has been assuming these a roughly equivalent in terms of performance (I mean logically they are the same right?) .

Benchmarking
a.c

#include <stdio.h>
#include <emscripten.h>

void foo(int param1, int param2)
{
}

typedef void (*fptr)(int param1, int param2);

EM_JS(int, callFunc, (fptr func, int param1, int param2), {

	// Do periodic testing, maybe a JIT only kicks in after having yielded back to main loop?
	setInterval(function() {
//		var t0 = process.hrtime();
		var t0 = performance.now();
		for(var i = 0; i < 1000000; ++i) {
			wasmTable.get(func)(param1, param2);
			wasmTable.get(func)(param1, param2);
			wasmTable.get(func)(param1, param2);
			wasmTable.get(func)(param1, param2);
			wasmTable.get(func)(param1, param2);
			wasmTable.get(func)(param1, param2);
			wasmTable.get(func)(param1, param2);
			wasmTable.get(func)(param1, param2);
		}
//		var t1 = process.hrtime();
		var t1 = performance.now();
//		console.log('8 million calls took ' + ((t1[0]-t0[0])*1e3 + (t1[1]-t0[1])/1e6) + ' msecs.');
		console.log('8 million calls took ' + (t1-t0) + ' msecs.');
	}, 1000);
});

int main()
{
	callFunc(foo, 42, 100);
}

with

emcc a.c -o a.html -O3

yields the following results:

Firefox 82
8 million calls took 679.2207000000001 msecs.
8 million calls took 678.0709999999999 msecs.
8 million calls took 682.4001000000007 msecs.

Node.js v12.18.1
8 million calls took 619.486099 msecs.
8 million calls took 617.083599 msecs.
8 million calls took 606.103299 msecs.

Chrome 86
8 million calls took 586.8250000057742 msecs.
8 million calls took 598.8599999982398 msecs.
8 million calls took 591.3149999978486 msecs.

Compared to static signature dispatch in Emscripten 2.0.1, dynCall_sig is

• 15.4x faster in Firefox 82,
• 6.6x faster in Node.js v12.18.1,
• 6.03x faster in Chrome 86
  than wasmTable.get(ptr).

It is noticeable though that wasmTable.get(ptr) is still an improvement in Firefox against the old dynCall('vii', func, [param1, param2]); that dynamically constructed arrays and did the .apply(null, [args]) stuff. In Node.js and Chrome that did not make as much effect. Thought the GC pressure was still likely there for them as well.

I would recommend that wasmTable.get() be used to solve dynamic signature dispatch and aggressive code size optimization problems only. In all cases, {{{ makeDynCall }}} should default to generating a dynCall_sig, unless -Oz is passed, in which case it would generate a wasmTable.get(). How does that sound?

[juj]

Interestingly, benchmarking

#include <stdio.h>
#include <emscripten.h>

void foo(int param1, int param2)
{
}

typedef void (*fptr)(int param1, int param2);

EM_JS(int, callFunc, (fptr func, int param1, int param2), {

	// Do periodic testing, maybe a JIT only kicks in after having yielded back to main loop?
	setInterval(function() {
		var t0 = process.hrtime();
//		var t0 = performance.now();
		var f = wasmTable.get(func);
		for(var i = 0; i < 1000000; ++i) {
			f(param1, param2);
			f(param1, param2);
			f(param1, param2);
			f(param1, param2);
			f(param1, param2);
			f(param1, param2);
			f(param1, param2);
			f(param1, param2);
		}
		var t1 = process.hrtime();
//		var t1 = performance.now();
		console.log('8 million calls took ' + ((t1[0]-t0[0])*1e3 + (t1[1]-t0[1])/1e6) + ' msecs.');
//		console.log('8 million calls took ' + (t1-t0) + ' msecs.');
	}, 1000);
});

int main()
{
	callFunc(foo, 42, 100);
}

gives

Firefox:
8 million calls took 28.97530000000006 msecs.
8 million calls took 29.52640000000065 msecs.
8 million calls took 28.96270000000004 msecs.

Node.js:
8 million calls took 75.1663 msecs.
8 million calls took 75.8751 msecs.
8 million calls took 76.1974 msecs.

Chrome:
8 million calls took 70.84999998915009 msecs.
8 million calls took 70.41000000026543 msecs.
8 million calls took 70.80999999016058 msecs.

with Firefox being super-fast. So in cases when one wants to call a function several times instead of just once, binding it beforehand via wasmTable.get() is the fastest way.

[sbc100]

Wow, thanks for benchmarking that. 6-15x slowdown over the wasm-side dynCalls is more than I would have imagined.

Do you think there are real world programs that make dynCalls frequently enough to be impacted by this slowdown though? How common are JS->wasm dyncalls in real world programs? @kripken do you think its work re-enabling the native dynCalls for optimizing builds?

[kripken]

Very interesting data! Good you found this @juj, we did not appreciate how big a difference this makes.

It looks like that supports that wasmTable.get() is the slowdown, and that it is not well-optimized in VMs compared to a call into the wasm to do a call_indirect. I remember there was an issue where wasmTable.get() needs to ensure a unique function for subsequent gets, so perhaps there is a lookup table or such happening?

It also looks like even in a loop the VM doesn't realize the gets will all return the same thing (which I guess they might not, if the wasm can modify the table, which in the future it might).

@jakobkummerow , thoughts on those?

The data also supports that caching the result of wasmTable.get is the most efficient thing, as then there is a direct call. That makes a lot of sense, and I think it's probably the thing worth focusing on most, as opposed to going back to calling into the wasm for a call_indirect all the time like dynCalls did.

One possibility is to mirror the table into a normal JS object. We do have full knowledge of when the table is modified (addFunction, removeFunction), at least barring pthreads+dynamic linking in the future. I tested with replacing wasmTable.get with

// A mirror of wasmTable.
var tableGetTable = [];

function tableGet(index) {
  // Grow the mirror if we need to.
  if (index >= tableGetTable.length) {
    tableGetTable.length = index + 1;
  }
  // Fetch from the table if we need to.
  if (!tableGetTable[index]) {
    tableGetTable[index] = wasmTable.get(index);
  }
  return tableGetTable[index];
}

That takes around 100ms for me, which is pretty close to what I get with just direct calls in the loop of a cached function (70, and which is definitely the upper limit, and not something we can likely achieve in general), and much faster than constantly calling wasmTable.get (450), and also faster than the dynCall approach of calling into the wasm to do a call_indirect (135).

The main downsides to this may be that it adds a little code size (at least fixed, not per-sig), and that it needs some integration with addFunction/removeFunction etc.

Thoughts?