Active Messages:

One application - use to solve the following game:
1. Marbles are placed in each of the "holes" except one.
2. A marble can be moved by crossing over an adjacent one to the next hole, thus eliminating the marble that was crossed.
3. Step 2 is repeated since only one marble remains.

One solution is to
1. Enumerate the moves to form a tree
2. Spread work over the nodes by applying a hash function on the board state and sending the job to the appropriate node.
3. Coalesce identical states.

Whether an alternative approach would be efficient: One approach that was proposed was to reverse the problem by starting with a single marble and enumerating the states backwards. But it was soon realised that this also boils down to the original problem if the holes are treated as marbles and vice-versa.

Trivia:
  Such a 5-sided triangle has solutions numbering ~1000
  For a 6-sided triangle  ~million
  7-sided triangle  0!
 

Optimistic Active Messages:
 RPC was demonstrated with OAM. The differences:
 RPC  - thread based; pre-emptive, schedulable, has own stack
AM - interrupt handler;  "steals" stack or uses special stack.

    so while interrupt handler is running, a thread can't use the stack - when there is a failure to acquire lock, a restart will be necessary.

Problem with RPC is possibility to fail to acquire lock.
One trade-off would be to start an AM handler, and fall over to RPC if we fail to acquire lock.

UNET:

Features:
Low latency
High bandwidth for small message sizes.

End-points are pinned memory regions on each side.

User process takes elements from free queue and inserts into send queue. NIC polls on send queue and sends it across the network, returning the free element to free queue.
At the receiver side, the exact opposite process takes place, with the NIC filling up the receive queue and the user process freeing it up.

Problem at receiver end - how do we know where to put the incoming packet ?

• Allocate an array of fixed worst case maximum message size blocks (phew!)
• Let NI allocate memory
• Remote write - sender says through packets where to put the message.

Costs of UNET:
 Multiple memory writes ( maintaining lists, copying messages, etc. )
 Why should the NI poll - why can't we use interrupts ?
  Lots of pinned memory regions - NI polls queue heads to see which queue is ready.
 
Reducing the costs:
 Only accelerate some connections
 Small message optimization - write data into queue entry directly ( this was implemented in ATM )

Where does UNET fit in the big picture ?

• Mux/De-mux: Done in end-point
• Notification:  Optional - polling, upcalls
• Buffering: Pinned memory areas
• DMA - to pinned areas
• Protocol acceleration: Hardware Xfers

 UNET: Demux -> handler
 Fugue: Handler -> demux

 Use of demux after handler helps integrating message delivery with VM.