[iximeow]

I'd found this useful in collecting some information (which I'll be including in a forthcoming Propolis issue) - @luqmana you may also find this useful, if you haven't put something similar together yourself 😅

buried in the output of this script, if you run something like perf bench futex wake, you will find gems like:

xapic t=2549993400124254 (vcpu=16) read: 4 bytes at fee00300
xapic t=2549993400130654 (vcpu=16) write: ICR_HI <- 12000000 (4 bytes)
xapic t=2549993400138935 (vcpu=16) write: ICR_LO <- fb (4 bytes)
xapic t=2549993400138935 (vcpu=16) ICR write: vector=251, destination=18 (physical)
    delivery mode 0x0 (Fixed) shorthand=0, status=de-assert, level=Idle, trigger=Edge
xapic t=2549993400175279 (vcpu=18) write: EOI Register <- 4 (0 bytes)
xapic t=2549993400235175 (vcpu=16) read: 4 bytes at fee00300
xapic t=2549993400239936 (vcpu=16) write: ICR_HI <- 12000000 (4 bytes)
xapic t=2549993400245477 (vcpu=16) write: ICR_LO <- fb (4 bytes)
xapic t=2549993400245477 (vcpu=16) ICR write: vector=251, destination=18 (physical)
    delivery mode 0x0 (Fixed) shorthand=0, status=de-assert, level=Idle, trigger=Edge
xapic t=2549993400262158 (vcpu=18) write: EOI Register <- 4 (0 bytes)

which is the APIC's view of doing a bunch of FUTEX_WAKEs in a Linx guest. namely, you can see that the ICR_HI write causes an exit, and it takes about 8.3 microseconds to get to ICR_LO and actually get to thinking about a cross-call to poke the target vCPU. the target (18) ends up on-core and writing to EOI another ~36 microseconds later, with the whole "notify the destination" coming out to a cool 50 microseconds or so. this is pretty consistent with what you'd see `perf bench futex wake' report in a guest.