{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"nim-libp2p documentation","text":"

Welcome to the nim-libp2p documentation!

Here, you'll find tutorials to help you get started, as well as the full reference.

"},{"location":"circuitrelay/","title":"Circuit Relay example","text":"

Circuit Relay can be used when a node cannot reach another node directly, but can reach it through another node (the Relay).

That may happen because of NAT, Firewalls, or incompatible transports.

More informations here.

import chronos, stew/byteutils\nimport libp2p, libp2p/protocols/connectivity/relay/[relay, client]\n\n# Helper to create a circuit relay node\nproc createCircuitRelaySwitch(r: Relay): Switch =\n  SwitchBuilder\n  .new()\n  .withRng(newRng())\n  .withAddresses(@[MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet()])\n  .withTcpTransport()\n  .withMplex()\n  .withNoise()\n  .withCircuitRelay(r)\n  .build()\n\nproc main() {.async.} =\n  # Create a custom protocol\n  let customProtoCodec = \"/test\"\n  var proto = new LPProtocol\n  proto.codec = customProtoCodec\n  proto.handler = proc(conn: Connection, proto: string) {.async.} =\n    var msg = string.fromBytes(await conn.readLp(1024))\n    echo \"1 - Dst Received: \", msg\n    assert \"test1\" == msg\n    await conn.writeLp(\"test2\")\n    msg = string.fromBytes(await conn.readLp(1024))\n    echo \"2 - Dst Received: \", msg\n    assert \"test3\" == msg\n    await conn.writeLp(\"test4\")\n\n  let\n    relay = Relay.new()\n    clSrc = RelayClient.new()\n    clDst = RelayClient.new()\n\n    # Create three hosts, enable relay client on two of them.\n    # The third one can relay connections for other peers.\n    # RelayClient can use a relay, Relay is a relay.\n    swRel = createCircuitRelaySwitch(relay)\n    swSrc = createCircuitRelaySwitch(clSrc)\n    swDst = createCircuitRelaySwitch(clDst)\n\n  swDst.mount(proto)\n\n  await swRel.start()\n  await swSrc.start()\n  await swDst.start()\n\n  let\n    # Create a relay address to swDst using swRel as the relay\n    addrs = MultiAddress\n      .init(\n        $swRel.peerInfo.addrs[0] & \"/p2p/\" & $swRel.peerInfo.peerId & \"/p2p-circuit\"\n      )\n      .get()\n\n  # Connect Dst to the relay\n  await swDst.connect(swRel.peerInfo.peerId, swRel.peerInfo.addrs)\n\n  # Dst reserve a slot on the relay.\n  let rsvp = await clDst.reserve(swRel.peerInfo.peerId, swRel.peerInfo.addrs)\n\n  # Src dial Dst using the relay\n  let conn = await swSrc.dial(swDst.peerInfo.peerId, @[addrs], customProtoCodec)\n\n  await conn.writeLp(\"test1\")\n  var msg = string.fromBytes(await conn.readLp(1024))\n  echo \"1 - Src Received: \", msg\n  assert \"test2\" == msg\n  await conn.writeLp(\"test3\")\n  msg = string.fromBytes(await conn.readLp(1024))\n  echo \"2 - Src Received: \", msg\n  assert \"test4\" == msg\n\n  await relay.stop()\n  await allFutures(swSrc.stop(), swDst.stop(), swRel.stop())\n\nwaitFor(main())\n
"},{"location":"tutorial_1_connect/","title":"Simple ping tutorial","text":"

Hi all, welcome to the first nim-libp2p tutorial!

This tutorial is for everyone who is interested in building peer-to-peer applications. No Nim programming experience is needed.

To give you a quick overview, Nim is the programming language we are using and nim-libp2p is the Nim implementation of libp2p, a modular library that enables the development of peer-to-peer network applications.

Hope you'll find it helpful in your journey of learning. Happy coding! ;)

"},{"location":"tutorial_1_connect/#before-you-start","title":"Before you start","text":"

The only prerequisite here is Nim, the programming language with a Python-like syntax and a performance similar to C. Detailed information can be found here.

Install Nim via their official website. Check Nim's installation via nim --version and its package manager Nimble via nimble --version.

You can now install the latest version of nim-libp2p: 

nimble install libp2p@#master\n

"},{"location":"tutorial_1_connect/#a-simple-ping-application","title":"A simple ping application","text":"

We'll start by creating a simple application, which is starting two libp2p switch, and pinging each other using the Ping protocol.

You can find the source of this tutorial (and other tutorials) in the libp2p/examples folder!

Let's create a part1.nim, and import our dependencies: 

import chronos\n\nimport libp2p\nimport libp2p/protocols/ping\n
chronos the asynchronous framework used by nim-libp2p

Next, we'll create a helper procedure to create our switches. A switch needs a bit of configuration, and it will be easier to do this configuration only once: 

proc createSwitch(ma: MultiAddress, rng: ref HmacDrbgContext): Switch =\n  var switch = SwitchBuilder\n    .new()\n    .withRng(rng)\n    # Give the application RNG\n    .withAddress(ma)\n    # Our local address(es)\n    .withTcpTransport()\n    # Use TCP as transport\n    .withMplex()\n    # Use Mplex as muxer\n    .withNoise()\n    # Use Noise as secure manager\n    .build()\n\n  return switch\n
This will create a switch using Mplex as a multiplexer, Noise to secure the communication, and TCP as an underlying transport.

You can of course tweak this, to use a different or multiple transport, or tweak the configuration of Mplex and Noise, but this is some sane defaults that we'll use going forward.

Let's now start to create our main procedure: 

proc main() {.async.} =\n  let\n    rng = newRng()\n    localAddress = MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet()\n    pingProtocol = Ping.new(rng = rng)\n
We created some variables that we'll need for the rest of the application: the global rng instance, our localAddress, and an instance of the Ping protocol. The address is in the MultiAddress format. The port 0 means \"take any port available\".

tryGet is procedure which is part of nim-result, that will throw an exception if the supplied MultiAddress is invalid.

We can now create our two switches: 

  let\n    switch1 = createSwitch(localAddress, rng)\n    switch2 = createSwitch(localAddress, rng)\n\n  switch1.mount(pingProtocol)\n\n  await switch1.start()\n  await switch2.start()\n
We've mounted the pingProtocol on our first switch. This means that the first switch will actually listen for any ping requests coming in, and handle them accordingly.

Now that we've started the nodes, they are listening for incoming peers. We can find out which port was attributed, and the resulting local addresses, by using switch1.peerInfo.addrs.

We'll dial the first switch from the second one, by specifying its Peer ID, its MultiAddress and the Ping protocol codec: 

  let conn =\n    await switch2.dial(switch1.peerInfo.peerId, switch1.peerInfo.addrs, PingCodec)\n
We now have a Ping connection setup between the second and the first switch, we can use it to actually ping the node: 
  # ping the other node and echo the ping duration\n  echo \"ping: \", await pingProtocol.ping(conn)\n\n  # We must close the connection ourselves when we're done with it\n  await conn.close()\n
And that's it! Just a little bit of cleanup: shutting down the switches, waiting for them to stop, and we'll call our main procedure: 
  await allFutures(switch1.stop(), switch2.stop())\n    # close connections and shutdown all transports\n\nwaitFor(main())\n
You can now run this program using nim c -r part1.nim, and you should see the dialing sequence, ending with a ping output.

In the next tutorial, we'll look at how to create our own custom protocol.

"},{"location":"tutorial_2_customproto/","title":"Custom protocol in libp2p","text":"

In the previous tutorial, we've looked at how to create a simple ping program using the nim-libp2p.

We'll now look at how to create a custom protocol inside the libp2p

Let's create a part2.nim, and import our dependencies: 

import chronos\nimport stew/byteutils\n\nimport libp2p\n
This is similar to the first tutorial, except we don't need to import the Ping protocol.

Next, we'll declare our custom protocol 

const TestCodec = \"/test/proto/1.0.0\"\n\ntype TestProto = ref object of LPProtocol\n
We've set a protocol ID, and created a custom LPProtocol. In a more complex protocol, we could use this structure to store interesting variables.

A protocol generally has two parts: a handling/server part, and a dialing/client part. These two parts can be identical, but in our trivial protocol, the server will wait for a message from the client, and the client will send a message, so we have to handle the two cases separately.

Let's start with the server part: 

proc new(T: typedesc[TestProto]): T =\n  # every incoming connections will in be handled in this closure\n  proc handle(conn: Connection, proto: string) {.async.} =\n    # Read up to 1024 bytes from this connection, and transform them into\n    # a string\n    echo \"Got from remote - \", string.fromBytes(await conn.readLp(1024))\n    # We must close the connections ourselves when we're done with it\n    await conn.close()\n\n  return T.new(codecs = @[TestCodec], handler = handle)\n
This is a constructor for our TestProto, that will specify our codecs and a handler, which will be called for each incoming peer asking for this protocol. In our handle, we simply read a message from the connection and echo it.

We can now create our client part: 

proc hello(p: TestProto, conn: Connection) {.async.} =\n  await conn.writeLp(\"Hello p2p!\")\n
Again, pretty straightforward, we just send a message on the connection.

We can now create our main procedure: 

proc main() {.async.} =\n  let\n    rng = newRng()\n    testProto = TestProto.new()\n    switch1 = newStandardSwitch(rng = rng)\n    switch2 = newStandardSwitch(rng = rng)\n\n  switch1.mount(testProto)\n\n  await switch1.start()\n  await switch2.start()\n\n  let conn =\n    await switch2.dial(switch1.peerInfo.peerId, switch1.peerInfo.addrs, TestCodec)\n\n  await testProto.hello(conn)\n\n  # We must close the connection ourselves when we're done with it\n  await conn.close()\n\n  await allFutures(switch1.stop(), switch2.stop())\n    # close connections and shutdown all transports\n
This is very similar to the first tutorial's main, the only noteworthy difference is that we use newStandardSwitch, which is similar to the createSwitch of the first tutorial, but is bundled directly in libp2p

We can now wrap our program by calling our main proc: 

waitFor(main())\n
And that's it! In the next tutorial, we'll create a more complex protocol using Protobuf.

"},{"location":"tutorial_3_protobuf/","title":"Protobuf usage","text":"

In the previous tutorial, we created a simple \"ping\" protocol. Most real protocol want their messages to be structured and extensible, which is why most real protocols use protobuf to define their message structures.

Here, we'll create a slightly more complex protocol, which parses & generate protobuf messages. Let's start by importing our dependencies, as usual: 

import chronos\nimport stew/results # for Opt[T]\n\nimport libp2p\n

"},{"location":"tutorial_3_protobuf/#protobuf-encoding-decoding","title":"Protobuf encoding & decoding","text":"

This will be the structure of our messages: 

message MetricList {\n  message Metric {\n    string name = 1;\n    float value = 2;\n  }\n\n  repeated Metric metrics = 2;\n}\n
We'll create our protobuf types, encoders & decoders, according to this format. To create the encoders & decoders, we are going to use minprotobuf (included in libp2p).

While more modern technics (such as nim-protobuf-serialization) exists, minprotobuf is currently the recommended method to handle protobuf, since it has been used in production extensively, and audited. 

type\n  Metric = object\n    name: string\n    value: float\n\n  MetricList = object\n    metrics: seq[Metric]\n\n{.push raises: [].}\n\nproc encode(m: Metric): ProtoBuffer =\n  result = initProtoBuffer()\n  result.write(1, m.name)\n  result.write(2, m.value)\n  result.finish()\n\nproc decode(_: type Metric, buf: seq[byte]): Result[Metric, ProtoError] =\n  var res: Metric\n  let pb = initProtoBuffer(buf)\n  # \"getField\" will return a Result[bool, ProtoError].\n  # The Result will hold an error if the protobuf is invalid.\n  # The Result will hold \"false\" if the field is missing\n  #\n  # We are just checking the error, and ignoring whether the value\n  # is present or not (default values are valid).\n  discard ?pb.getField(1, res.name)\n  discard ?pb.getField(2, res.value)\n  ok(res)\n\nproc encode(m: MetricList): ProtoBuffer =\n  result = initProtoBuffer()\n  for metric in m.metrics:\n    result.write(1, metric.encode())\n  result.finish()\n\nproc decode(_: type MetricList, buf: seq[byte]): Result[MetricList, ProtoError] =\n  var\n    res: MetricList\n    metrics: seq[seq[byte]]\n  let pb = initProtoBuffer(buf)\n  discard ?pb.getRepeatedField(1, metrics)\n\n  for metric in metrics:\n    res.metrics &= ?Metric.decode(metric)\n  ok(res)\n

"},{"location":"tutorial_3_protobuf/#results-instead-of-exceptions","title":"Results instead of exceptions","text":"

As you can see, this part of the program also uses Results instead of exceptions for error handling. We start by {.push raises: [].}, which will prevent every non-async function from raising exceptions.

Then, we use nim-result to convey errors to function callers. A Result[T, E] will either hold a valid result of type T, or an error of type E.

You can check if the call succeeded by using res.isOk, and then get the value using res.value or the error by using res.error.

Another useful tool is ?, which will unpack a Result if it succeeded, or if it failed, exit the current procedure returning the error.

nim-result is packed with other functionalities that you'll find in the nim-result repository.

Results and exception are generally interchangeable, but have different semantics that you may or may not prefer.

"},{"location":"tutorial_3_protobuf/#creating-the-protocol","title":"Creating the protocol","text":"

We'll next create a protocol, like in the last tutorial, to request these metrics from our host 

type\n  MetricCallback = proc(): Future[MetricList] {.raises: [], gcsafe.}\n  MetricProto = ref object of LPProtocol\n    metricGetter: MetricCallback\n\nproc new(_: typedesc[MetricProto], cb: MetricCallback): MetricProto =\n  var res: MetricProto\n  proc handle(conn: Connection, proto: string) {.async.} =\n    let\n      metrics = await res.metricGetter()\n      asProtobuf = metrics.encode()\n    await conn.writeLp(asProtobuf.buffer)\n    await conn.close()\n\n  res = MetricProto.new(@[\"/metric-getter/1.0.0\"], handle)\n  res.metricGetter = cb\n  return res\n\nproc fetch(p: MetricProto, conn: Connection): Future[MetricList] {.async.} =\n  let protobuf = await conn.readLp(2048)\n  # tryGet will raise an exception if the Result contains an error.\n  # It's useful to bridge between exception-world and result-world\n  return MetricList.decode(protobuf).tryGet()\n
We can now create our main procedure: 
proc main() {.async.} =\n  let rng = newRng()\n  proc randomMetricGenerator(): Future[MetricList] {.async.} =\n    let metricCount = rng[].generate(uint32) mod 16\n    for i in 0 ..< metricCount + 1:\n      result.metrics.add(\n        Metric(name: \"metric_\" & $i, value: float(rng[].generate(uint16)) / 1000.0)\n      )\n    return result\n\n  let\n    metricProto1 = MetricProto.new(randomMetricGenerator)\n    metricProto2 = MetricProto.new(randomMetricGenerator)\n    switch1 = newStandardSwitch(rng = rng)\n    switch2 = newStandardSwitch(rng = rng)\n\n  switch1.mount(metricProto1)\n\n  await switch1.start()\n  await switch2.start()\n\n  let\n    conn = await switch2.dial(\n      switch1.peerInfo.peerId, switch1.peerInfo.addrs, metricProto2.codecs\n    )\n    metrics = await metricProto2.fetch(conn)\n  await conn.close()\n\n  for metric in metrics.metrics:\n    echo metric.name, \" = \", metric.value\n\n  await allFutures(switch1.stop(), switch2.stop())\n    # close connections and shutdown all transports\n\nwaitFor(main())\n
If you run this program, you should see random metrics being sent from the switch1 to the switch2.

"},{"location":"tutorial_4_gossipsub/","title":"GossipSub","text":"

In this tutorial, we'll build a simple GossipSub network to broadcast the metrics we built in the previous tutorial.

GossipSub is used to broadcast some messages in a network, and allows to balance between latency, bandwidth usage, privacy and attack resistance.

You'll find a good explanation of how GossipSub works here. There are a lot of parameters you can tweak to adjust how GossipSub behaves but here we'll use the sane defaults shipped with libp2p.

We'll start by creating our metric structure like previously 

import chronos\nimport stew/results\n\nimport libp2p\nimport libp2p/protocols/pubsub/rpc/messages\n\ntype\n  Metric = object\n    name: string\n    value: float\n\n  MetricList = object\n    hostname: string\n    metrics: seq[Metric]\n\n{.push raises: [].}\n\nproc encode(m: Metric): ProtoBuffer =\n  result = initProtoBuffer()\n  result.write(1, m.name)\n  result.write(2, m.value)\n  result.finish()\n\nproc decode(_: type Metric, buf: seq[byte]): Result[Metric, ProtoError] =\n  var res: Metric\n  let pb = initProtoBuffer(buf)\n  discard ?pb.getField(1, res.name)\n  discard ?pb.getField(2, res.value)\n  ok(res)\n\nproc encode(m: MetricList): ProtoBuffer =\n  result = initProtoBuffer()\n  for metric in m.metrics:\n    result.write(1, metric.encode())\n  result.write(2, m.hostname)\n  result.finish()\n\nproc decode(_: type MetricList, buf: seq[byte]): Result[MetricList, ProtoError] =\n  var\n    res: MetricList\n    metrics: seq[seq[byte]]\n  let pb = initProtoBuffer(buf)\n  discard ?pb.getRepeatedField(1, metrics)\n\n  for metric in metrics:\n    res.metrics &= ?Metric.decode(metric)\n  ?pb.getRequiredField(2, res.hostname)\n  ok(res)\n
This is exactly like the previous structure, except that we added a hostname to distinguish where the metric is coming from.

Now we'll create a small GossipSub network to broadcast the metrics, and collect them on one of the node. 

type Node = tuple[switch: Switch, gossip: GossipSub, hostname: string]\n\nproc oneNode(node: Node, rng: ref HmacDrbgContext) {.async.} =\n  # This procedure will handle one of the node of the network\n  node.gossip.addValidator(\n    [\"metrics\"],\n    proc(topic: string, message: Message): Future[ValidationResult] {.async.} =\n      let decoded = MetricList.decode(message.data)\n      if decoded.isErr:\n        return ValidationResult.Reject\n      return ValidationResult.Accept\n    ,\n  )\n  # This \"validator\" will attach to the `metrics` topic and make sure\n  # that every message in this topic is valid. This allows us to stop\n  # propagation of invalid messages quickly in the network, and punish\n  # peers sending them.\n\n  # `John` will be responsible to log the metrics, the rest of the nodes\n  # will just forward them in the network\n  if node.hostname == \"John\":\n    node.gossip.subscribe(\n      \"metrics\",\n      proc(topic: string, data: seq[byte]) {.async.} =\n        echo MetricList.decode(data).tryGet()\n      ,\n    )\n  else:\n    node.gossip.subscribe(\"metrics\", nil)\n\n  # Create random metrics 10 times and broadcast them\n  for _ in 0 ..< 10:\n    await sleepAsync(500.milliseconds)\n    var metricList = MetricList(hostname: node.hostname)\n    let metricCount = rng[].generate(uint32) mod 4\n    for i in 0 ..< metricCount + 1:\n      metricList.metrics.add(\n        Metric(name: \"metric_\" & $i, value: float(rng[].generate(uint16)) / 1000.0)\n      )\n\n    discard await node.gossip.publish(\"metrics\", encode(metricList).buffer)\n  await node.switch.stop()\n
For our main procedure, we'll create a few nodes, and connect them together. Note that they are not all interconnected, but GossipSub will take care of broadcasting to the full network nonetheless. 
proc main() {.async.} =\n  let rng = newRng()\n  var nodes: seq[Node]\n\n  for hostname in [\"John\", \"Walter\", \"David\", \"Thuy\", \"Amy\"]:\n    let\n      switch = newStandardSwitch(rng = rng)\n      gossip = GossipSub.init(switch = switch, triggerSelf = true)\n    switch.mount(gossip)\n    await switch.start()\n\n    nodes.add((switch, gossip, hostname))\n\n  for index, node in nodes:\n    # Connect to a few neighbors\n    for otherNodeIdx in index - 1 .. index + 2:\n      if otherNodeIdx notin 0 ..< nodes.len or otherNodeIdx == index:\n        continue\n      let otherNode = nodes[otherNodeIdx]\n      await node.switch.connect(\n        otherNode.switch.peerInfo.peerId, otherNode.switch.peerInfo.addrs\n      )\n\n  var allFuts: seq[Future[void]]\n  for node in nodes:\n    allFuts.add(oneNode(node, rng))\n\n  await allFutures(allFuts)\n\nwaitFor(main())\n
If you run this program, you should see something like: 
(hostname: \"John\", metrics: @[(name: \"metric_0\", value: 42.097), (name: \"metric_1\", value: 50.99), (name: \"metric_2\", value: 47.86), (name: \"metric_3\", value: 5.368)])\n(hostname: \"Walter\", metrics: @[(name: \"metric_0\", value: 39.452), (name: \"metric_1\", value: 15.606), (name: \"metric_2\", value: 14.059), (name: \"metric_3\", value: 6.68)])\n(hostname: \"David\", metrics: @[(name: \"metric_0\", value: 9.82), (name: \"metric_1\", value: 2.862), (name: \"metric_2\", value: 15.514)])\n(hostname: \"Thuy\", metrics: @[(name: \"metric_0\", value: 59.038)])\n(hostname: \"Amy\", metrics: @[(name: \"metric_0\", value: 55.616), (name: \"metric_1\", value: 23.52), (name: \"metric_2\", value: 59.081), (name: \"metric_3\", value: 2.516)])\n

This is John receiving & logging everyone's metrics.

"},{"location":"tutorial_4_gossipsub/#going-further","title":"Going further","text":"

Building efficient & safe GossipSub networks is a tricky subject. By tweaking the gossip params and topic params, you can achieve very different properties.

Also see reports for GossipSub v1.1

If you are interested in broadcasting for your application, you may want to use Waku, which builds on top of GossipSub, and adds features such as history, spam protection, and light node friendliness.

"},{"location":"tutorial_5_discovery/","title":"Discovery Manager","text":"

In the previous tutorial, we built a custom protocol using protobuf and spread informations (some metrics) on the network using gossipsub. For this tutorial, on the other hand, we'll go back to a simple example we'll try to discover a specific peers to greet on the network.

First, as usual, we import the dependencies: 

import sequtils\nimport chronos\nimport stew/byteutils\n\nimport libp2p\nimport libp2p/protocols/rendezvous\nimport libp2p/discovery/rendezvousinterface\nimport libp2p/discovery/discoverymngr\n
We'll not use newStandardSwitch this time as we need the discovery protocol RendezVous to be mounted on the switch using withRendezVous.

Note that other discovery methods such as Kademlia or discv5 exist. 

proc createSwitch(rdv: RendezVous = RendezVous.new()): Switch =\n  SwitchBuilder\n  .new()\n  .withRng(newRng())\n  .withAddresses(@[MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet()])\n  .withTcpTransport()\n  .withYamux()\n  .withNoise()\n  .withRendezVous(rdv)\n  .build()\n\n# Create a really simple protocol to log one message received then close the stream\nconst DumbCodec = \"/dumb/proto/1.0.0\"\ntype DumbProto = ref object of LPProtocol\nproc new(T: typedesc[DumbProto], nodeNumber: int): T =\n  proc handle(conn: Connection, proto: string) {.async.} =\n    echo \"Node\", nodeNumber, \" received: \", string.fromBytes(await conn.readLp(1024))\n    await conn.close()\n\n  return T.new(codecs = @[DumbCodec], handler = handle)\n

"},{"location":"tutorial_5_discovery/#bootnodes","title":"Bootnodes","text":"

The first time a p2p program is ran, he needs to know how to join its network. This is generally done by hard-coding a list of stable nodes in the binary, called \"bootnodes\". These bootnodes are a critical part of a p2p network, since they are used by every new user to onboard the network.

By using libp2p, we can use any node supporting our discovery protocol (rendezvous in this case) as a bootnode. For this example, we'll create a bootnode, and then every peer will advertise itself on the bootnode, and use it to find other peers 

proc main() {.async.} =\n  let bootNode = createSwitch()\n  await bootNode.start()\n\n  # Create 5 nodes in the network\n  var\n    switches: seq[Switch] = @[]\n    discManagers: seq[DiscoveryManager] = @[]\n\n  for i in 0 .. 5:\n    let rdv = RendezVous.new()\n    # Create a remote future to await at the end of the program\n    let switch = createSwitch(rdv)\n    switch.mount(DumbProto.new(i))\n    switches.add(switch)\n\n    # A discovery manager is a simple tool, you can set it up by adding discovery\n    # interfaces (such as RendezVousInterface) then you can use it to advertise\n    # something on the network or to request something from it.\n    let dm = DiscoveryManager()\n    # A RendezVousInterface is a RendezVous protocol wrapped to be usable by the\n    # DiscoveryManager.\n    dm.add(RendezVousInterface.new(rdv))\n    discManagers.add(dm)\n\n    # We can now start the switch and connect to the bootnode\n    await switch.start()\n    await switch.connect(bootNode.peerInfo.peerId, bootNode.peerInfo.addrs)\n\n    # Each nodes of the network will advertise on some topics (EvenGang or OddClub)\n    dm.advertise(RdvNamespace(if i mod 2 == 0: \"EvenGang\" else: \"OddClub\"))\n
We can now create the newcomer. This peer will connect to the boot node, and use it to discover peers & greet them.

  let\n    rdv = RendezVous.new()\n    newcomer = createSwitch(rdv)\n    dm = DiscoveryManager()\n  await newcomer.start()\n  await newcomer.connect(bootNode.peerInfo.peerId, bootNode.peerInfo.addrs)\n  dm.add(RendezVousInterface.new(rdv, ttr = 250.milliseconds))\n\n  # Use the discovery manager to find peers on the OddClub topic to greet them\n  let queryOddClub = dm.request(RdvNamespace(\"OddClub\"))\n  for _ in 0 .. 2:\n    let\n      # getPeer give you a PeerAttribute containing informations about the peer.\n      res = await queryOddClub.getPeer()\n      # Here we will use the PeerId and the MultiAddress to greet him\n      conn = await newcomer.dial(res[PeerId], res.getAll(MultiAddress), DumbCodec)\n    await conn.writeLp(\"Odd Club suuuucks! Even Gang is better!\")\n    # Uh-oh!\n    await conn.close()\n    # Wait for the peer to close the stream\n    await conn.join()\n  # Queries will run in a loop, so we must stop them when we are done\n  queryOddClub.stop()\n\n  # Maybe it was because he wanted to join the EvenGang\n  let queryEvenGang = dm.request(RdvNamespace(\"EvenGang\"))\n  for _ in 0 .. 2:\n    let\n      res = await queryEvenGang.getPeer()\n      conn = await newcomer.dial(res[PeerId], res.getAll(MultiAddress), DumbCodec)\n    await conn.writeLp(\"Even Gang is sooo laaame! Odd Club rocks!\")\n    # Or maybe not...\n    await conn.close()\n    await conn.join()\n  queryEvenGang.stop()\n  # What can I say, some people just want to watch the world burn... Anyway\n\n  # Stop all the discovery managers\n  for d in discManagers:\n    d.stop()\n  dm.stop()\n\n  # Stop all the switches\n  await allFutures(switches.mapIt(it.stop()))\n  await allFutures(bootNode.stop(), newcomer.stop())\n\nwaitFor(main())\n
"},{"location":"tutorial_6_game/","title":"Tron example","text":"

In this tutorial, we will create a video game based on libp2p, using all of the features we talked about in the last tutorials.

We will: - Discover peers using the Discovery Manager - Use GossipSub to find a play mate - Create a custom protocol to play with him

While this may look like a daunting project, it's less than 150 lines of code.

The game will be a simple Tron. We will use nico as a game engine. (you need to run nimble install nico to have it available)

We will start by importing our dependencies and creating our types 

import os\nimport nico, chronos, stew/byteutils, stew/endians2\nimport libp2p\nimport libp2p/protocols/rendezvous\nimport libp2p/discovery/rendezvousinterface\nimport libp2p/discovery/discoverymngr\n\nconst\n  directions = @[(K_UP, 0, -1), (K_LEFT, -1, 0), (K_DOWN, 0, 1), (K_RIGHT, 1, 0)]\n  mapSize = 32\n  tickPeriod = 0.2\n\ntype\n  Player = ref object\n    x, y: int\n    currentDir, nextDir: int\n    lost: bool\n    color: int\n\n  Game = ref object\n    gameMap: array[mapSize * mapSize, int]\n    tickTime: float\n    localPlayer, remotePlayer: Player\n    peerFound: Future[Connection]\n    hasCandidate: bool\n    tickFinished: Future[int]\n\n  GameProto = ref object of LPProtocol\n\nproc new(_: type[Game]): Game =\n  # Default state of a game\n  result = Game(\n    tickTime: -3.0, # 3 seconds of \"warm-up\" time\n    localPlayer: Player(x: 4, y: 16, currentDir: 3, nextDir: 3, color: 8),\n    remotePlayer: Player(x: 27, y: 16, currentDir: 1, nextDir: 1, color: 12),\n    peerFound: newFuture[Connection](),\n  )\n  for pos in 0 .. result.gameMap.high:\n    if pos mod mapSize in [0, mapSize - 1] or pos div mapSize in [0, mapSize - 1]:\n      result.gameMap[pos] = 7\n

"},{"location":"tutorial_6_game/#game-logic","title":"Game Logic","text":"

The networking during the game will work like this:

This is a very simplistic scheme, but creating proper networking for video games is an art

The main drawback of this scheme is that the more ping you have with the peer, the slower the game will run. Or invertedly, the less ping you have, the faster it runs! 

proc update(g: Game, dt: float32) =\n  # Will be called at each frame of the game.\n  #\n  # Because both Nico and Chronos have a main loop,\n  # they must share the control of the main thread.\n  # This is a hacky way to make this happen\n  waitFor(sleepAsync(1.milliseconds))\n  # Don't do anything if we are still waiting for an opponent\n  if not (g.peerFound.finished()) or isNil(g.tickFinished):\n    return\n  g.tickTime += dt\n\n  # Update the wanted direction, making sure we can't go backward\n  for i in 0 .. directions.high:\n    if i != (g.localPlayer.currentDir + 2 mod 4) and keyp(directions[i][0]):\n      g.localPlayer.nextDir = i\n\n  if g.tickTime > tickPeriod and not g.tickFinished.finished():\n    # We choosen our next direction, let the networking know\n    g.localPlayer.currentDir = g.localPlayer.nextDir\n    g.tickFinished.complete(g.localPlayer.currentDir)\n\nproc tick(g: Game, p: Player) =\n  # Move player and check if he lost\n  p.x += directions[p.currentDir][1]\n  p.y += directions[p.currentDir][2]\n  if g.gameMap[p.y * mapSize + p.x] != 0:\n    p.lost = true\n  g.gameMap[p.y * mapSize + p.x] = p.color\n\nproc mainLoop(g: Game, peer: Connection) {.async.} =\n  while not (g.localPlayer.lost or g.remotePlayer.lost):\n    if g.tickTime > 0.0:\n      g.tickTime = 0\n    g.tickFinished = newFuture[int]()\n\n    # Wait for a choosen direction\n    let dir = await g.tickFinished\n    # Send it\n    await peer.writeLp(toBytes(uint32(dir)))\n\n    # Get the one from the peer\n    g.remotePlayer.currentDir = int uint32.fromBytes(await peer.readLp(8))\n    # Tick the players & restart\n    g.tick(g.remotePlayer)\n    g.tick(g.localPlayer)\n
We'll draw the map & put some texts when necessary: 
proc draw(g: Game) =\n  for pos, color in g.gameMap:\n    setColor(color)\n    boxFill(pos mod 32 * 4, pos div 32 * 4, 4, 4)\n  let text =\n    if not (g.peerFound.finished()):\n      \"Matchmaking..\"\n    elif g.tickTime < -1.5:\n      \"Welcome to Etron\"\n    elif g.tickTime < 0.0:\n      \"- \" & $(int(abs(g.tickTime) / 0.5) + 1) & \" -\"\n    elif g.remotePlayer.lost and g.localPlayer.lost:\n      \"DEUCE\"\n    elif g.localPlayer.lost:\n      \"YOU LOOSE\"\n    elif g.remotePlayer.lost:\n      \"YOU WON\"\n    else:\n      \"\"\n  printc(text, screenWidth div 2, screenHeight div 2)\n

"},{"location":"tutorial_6_game/#matchmaking","title":"Matchmaking","text":"

To find an opponent, we will broadcast our address on a GossipSub topic, and wait for someone to connect to us. We will also listen to that topic, and connect to anyone broadcasting his address.

If we are looking for a game, we'll send ok to let the peer know that we are available, check that he is also available, and launch the game. 

proc new(T: typedesc[GameProto], g: Game): T =\n  proc handle(conn: Connection, proto: string) {.async.} =\n    defer:\n      await conn.closeWithEof()\n    if g.peerFound.finished or g.hasCandidate:\n      await conn.close()\n      return\n    g.hasCandidate = true\n    await conn.writeLp(\"ok\")\n    if \"ok\" != string.fromBytes(await conn.readLp(1024)):\n      g.hasCandidate = false\n      return\n    g.peerFound.complete(conn)\n    # The handler of a protocol must wait for the stream to\n    # be finished before returning\n    await conn.join()\n\n  return T.new(codecs = @[\"/tron/1.0.0\"], handler = handle)\n\nproc networking(g: Game) {.async.} =\n  # Create our switch, similar to the GossipSub example and\n  # the Discovery examples combined\n  let\n    rdv = RendezVous.new()\n    switch = SwitchBuilder\n      .new()\n      .withRng(newRng())\n      .withAddresses(@[MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet()])\n      .withTcpTransport()\n      .withYamux()\n      .withNoise()\n      .withRendezVous(rdv)\n      .build()\n    dm = DiscoveryManager()\n    gameProto = GameProto.new(g)\n    gossip = GossipSub.init(switch = switch, triggerSelf = false)\n  dm.add(RendezVousInterface.new(rdv))\n\n  switch.mount(gossip)\n  switch.mount(gameProto)\n\n  gossip.subscribe(\n    \"/tron/matchmaking\",\n    proc(topic: string, data: seq[byte]) {.async.} =\n      # If we are still looking for an opponent,\n      # try to match anyone broadcasting its address\n      if g.peerFound.finished or g.hasCandidate:\n        return\n      g.hasCandidate = true\n\n      try:\n        let\n          (peerId, multiAddress) = parseFullAddress(data).tryGet()\n          stream = await switch.dial(peerId, @[multiAddress], gameProto.codec)\n\n        await stream.writeLp(\"ok\")\n        if (await stream.readLp(10)) != \"ok\".toBytes:\n          g.hasCandidate = false\n          return\n        g.peerFound.complete(stream)\n        # We are \"player 2\"\n        swap(g.localPlayer, g.remotePlayer)\n      except CatchableError as exc:\n        discard\n    ,\n  )\n\n  await switch.start()\n  defer:\n    await switch.stop()\n\n  # As explained in the last tutorial, we need a bootnode to be able\n  # to find peers. We could use any libp2p running rendezvous (or any\n  # node running tron). We will take it's MultiAddress from the command\n  # line parameters\n  if paramCount() > 0:\n    let (peerId, multiAddress) = paramStr(1).parseFullAddress().tryGet()\n    await switch.connect(peerId, @[multiAddress])\n  else:\n    echo \"No bootnode provided, listening on: \", switch.peerInfo.fullAddrs.tryGet()\n\n  # Discover peers from the bootnode, and connect to them\n  dm.advertise(RdvNamespace(\"tron\"))\n  let discoveryQuery = dm.request(RdvNamespace(\"tron\"))\n  discoveryQuery.forEach:\n    try:\n      await switch.connect(peer[PeerId], peer.getAll(MultiAddress))\n    except CatchableError as exc:\n      echo \"Failed to dial a peer: \", exc.msg\n\n  # We will try to publish our address multiple times, in case\n  # it takes time to establish connections with other GossipSub peers\n  var published = false\n  while not published:\n    await sleepAsync(500.milliseconds)\n    for fullAddr in switch.peerInfo.fullAddrs.tryGet():\n      if (await gossip.publish(\"/tron/matchmaking\", fullAddr.bytes)) == 0:\n        published = false\n        break\n      published = true\n\n  discoveryQuery.stop()\n\n  # We now wait for someone to connect to us (or for us to connect to someone)\n  let peerConn = await g.peerFound\n  defer:\n    await peerConn.closeWithEof()\n\n  await g.mainLoop(peerConn)\n\nlet\n  game = Game.new()\n  netFut = networking(game)\nnico.init(\"Status\", \"Tron\")\nnico.createWindow(\"Tron\", mapSize * 4, mapSize * 4, 4, false)\nnico.run(\n  proc() =\n    discard\n  ,\n  proc(dt: float32) =\n    game.update(dt)\n  ,\n  proc() =\n    game.draw()\n  ,\n)\nwaitFor(netFut.cancelAndWait())\n
And that's it! If you want to run this code locally, the simplest way is to use the first node as a boot node for the second one. But you can also use any rendezvous node

"},{"location":"go-daemon/daemonapi/","title":"Table of Contents","text":""},{"location":"go-daemon/daemonapi/#introduction","title":"Introduction","text":"

This is a libp2p-backed daemon wrapping the functionalities of go-libp2p for use in Nim. For more information about the go daemon, check out this repository.

Required only for running the tests.

"},{"location":"go-daemon/daemonapi/#prerequisites","title":"Prerequisites","text":"

Go with version 1.15.15.

You will likely be able to build go-libp2p-daemon with different Go versions, but they haven't been tested.

"},{"location":"go-daemon/daemonapi/#installation","title":"Installation","text":"

Follow one of the methods below:

"},{"location":"go-daemon/daemonapi/#script","title":"Script","text":"

Run the build script while having the go command pointing to the correct Go version. We recommend using 1.15.15, as previously stated. 

./scripts/build_p2pd.sh\n
If everything goes correctly, the binary (p2pd) should be built and placed in the correct directory. If you find any issues, please head into our discord and ask for our assistance.

After successfully building the binary, remember to add it to your path so it can be found. You can do that by running: 

export PATH=\"$PATH:$HOME/go/bin\"\n

Tip: To make this change permanent, add the command above to your .bashrc file.

"},{"location":"go-daemon/daemonapi/#usage","title":"Usage","text":""},{"location":"go-daemon/daemonapi/#example","title":"Example","text":"

Examples can be found in the examples folder

"},{"location":"go-daemon/daemonapi/#getting-started","title":"Getting Started","text":"

Try out the chat example. Full code can be found here:

nim c -r --threads:on examples/directchat.nim\n

This will output a peer ID such as QmbmHfVvouKammmQDJck4hz33WvVktNEe7pasxz2HgseRu which you can use in another instance to connect to it.

./examples/directchat\n/connect QmbmHfVvouKammmQDJck4hz33WvVktNEe7pasxz2HgseRu\n

You can now chat between the instances!

"}]}