1 line
37 KiB
JSON
1 line
37 KiB
JSON
{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"nim-libp2p documentation","text":"<p>Welcome to the nim-libp2p documentation!</p> <p>Here, you'll find tutorials to help you get started, as well as the full reference.</p>"},{"location":"circuitrelay/","title":"Circuit Relay example","text":"<p>Circuit Relay can be used when a node cannot reach another node directly, but can reach it through a another node (the Relay).</p> <p>That may happen because of NAT, Firewalls, or incompatible transports.</p> <p>More informations here.</p> <pre><code>import chronos, stew/byteutils\nimport libp2p,\nlibp2p/protocols/connectivity/relay/[relay, client]\n# Helper to create a circuit relay node\nproc createCircuitRelaySwitch(r: Relay): Switch =\nSwitchBuilder.new()\n.withRng(newRng())\n.withAddresses(@[ MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet() ])\n.withTcpTransport()\n.withMplex()\n.withNoise()\n.withCircuitRelay(r)\n.build()\nproc main() {.async.} =\n# Create a custom protocol\nlet customProtoCodec = \"/test\"\nvar proto = new LPProtocol\nproto.codec = customProtoCodec\nproto.handler = proc(conn: Connection, proto: string) {.async.} =\nvar msg = string.fromBytes(await conn.readLp(1024))\necho \"1 - Dst Received: \", msg\nassert \"test1\" == msg\nawait conn.writeLp(\"test2\")\nmsg = string.fromBytes(await conn.readLp(1024))\necho \"2 - Dst Received: \", msg\nassert \"test3\" == msg\nawait conn.writeLp(\"test4\")\nlet\nrelay = Relay.new()\nclSrc = RelayClient.new()\nclDst = RelayClient.new()\n# Create three hosts, enable relay client on two of them.\n# The third one can relay connections for other peers.\n# RelayClient can use a relay, Relay is a relay.\nswRel = createCircuitRelaySwitch(relay)\nswSrc = createCircuitRelaySwitch(clSrc)\nswDst = createCircuitRelaySwitch(clDst)\nswDst.mount(proto)\nawait swRel.start()\nawait swSrc.start()\nawait swDst.start()\nlet\n# Create a relay address to swDst using swRel as the relay\naddrs = MultiAddress.init($swRel.peerInfo.addrs[0] & \"/p2p/\" &\n$swRel.peerInfo.peerId & \"/p2p-circuit\").get()\n# Connect Dst to the relay\nawait swDst.connect(swRel.peerInfo.peerId, swRel.peerInfo.addrs)\n# Dst reserve a slot on the relay.\nlet rsvp = await clDst.reserve(swRel.peerInfo.peerId, swRel.peerInfo.addrs)\n# Src dial Dst using the relay\nlet conn = await swSrc.dial(swDst.peerInfo.peerId, @[ addrs ], customProtoCodec)\nawait conn.writeLp(\"test1\")\nvar msg = string.fromBytes(await conn.readLp(1024))\necho \"1 - Src Received: \", msg\nassert \"test2\" == msg\nawait conn.writeLp(\"test3\")\nmsg = string.fromBytes(await conn.readLp(1024))\necho \"2 - Src Received: \", msg\nassert \"test4\" == msg\nawait relay.stop()\nawait allFutures(swSrc.stop(), swDst.stop(), swRel.stop())\nwaitFor(main())\n</code></pre>"},{"location":"tutorial_1_connect/","title":"Simple ping tutorial","text":"<p>Hi all, welcome to the first nim-libp2p tutorial!</p> <p>This tutorial is for everyone who is interested in building peer-to-peer applications. No Nim programming experience is needed.</p> <p>To give you a quick overview, Nim is the programming language we are using and nim-libp2p is the Nim implementation of libp2p, a modular library that enables the development of peer-to-peer network applications.</p> <p>Hope you'll find it helpful in your journey of learning. Happy coding! ;)</p>"},{"location":"tutorial_1_connect/#before-you-start","title":"Before you start","text":"<p>The only prerequisite here is Nim, the programming language with a Python-like syntax and a performance similar to C. Detailed information can be found here.</p> <p>Install Nim via their official website. Check Nim's installation via <code>nim --version</code> and its package manager Nimble via <code>nimble --version</code>.</p> <p>You can now install the latest version of <code>nim-libp2p</code>: <pre><code>nimble install libp2p@#master\n</code></pre></p>"},{"location":"tutorial_1_connect/#a-simple-ping-application","title":"A simple ping application","text":"<p>We'll start by creating a simple application, which is starting two libp2p switch, and pinging each other using the Ping protocol.</p> <p>You can find the source of this tutorial (and other tutorials) in the libp2p/examples folder!</p> <p>Let's create a <code>part1.nim</code>, and import our dependencies: <pre><code>import chronos\nimport libp2p\nimport libp2p/protocols/ping\n</code></pre> chronos the asynchronous framework used by <code>nim-libp2p</code></p> <p>Next, we'll create an helper procedure to create our switches. A switch needs a bit of configuration, and it will be easier to do this configuration only once: <pre><code>proc createSwitch(ma: MultiAddress, rng: ref HmacDrbgContext): Switch =\nvar switch = SwitchBuilder\n.new()\n.withRng(rng) # Give the application RNG\n.withAddress(ma) # Our local address(es)\n.withTcpTransport() # Use TCP as transport\n.withMplex() # Use Mplex as muxer\n.withNoise() # Use Noise as secure manager\n.build()\nreturn switch\n</code></pre> This will create a switch using Mplex as a multiplexer, Noise to secure the communication, and TCP as an underlying transport.</p> <p>You can of course tweak this, to use a different or multiple transport, or tweak the configuration of Mplex and Noise, but this is some sane defaults that we'll use going forward.</p> <p>Let's now start to create our main procedure: <pre><code>proc main() {.async, gcsafe.} =\nlet\nrng = newRng()\nlocalAddress = MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet()\npingProtocol = Ping.new(rng=rng)\n</code></pre> We created some variables that we'll need for the rest of the application: the global <code>rng</code> instance, our <code>localAddress</code>, and an instance of the <code>Ping</code> protocol. The address is in the MultiAddress format. The port <code>0</code> means \"take any port available\".</p> <p><code>tryGet</code> is procedure which is part of nim-result, that will throw an exception if the supplied MultiAddress is invalid.</p> <p>We can now create our two switches: <pre><code> let\nswitch1 = createSwitch(localAddress, rng)\nswitch2 = createSwitch(localAddress, rng)\nswitch1.mount(pingProtocol)\nawait switch1.start()\nawait switch2.start()\n</code></pre> We've mounted the <code>pingProtocol</code> on our first switch. This means that the first switch will actually listen for any ping requests coming in, and handle them accordingly.</p> <p>Now that we've started the nodes, they are listening for incoming peers. We can find out which port was attributed, and the resulting local addresses, by using <code>switch1.peerInfo.addrs</code>.</p> <p>We'll dial the first switch from the second one, by specifying it's Peer ID, it's MultiAddress and the <code>Ping</code> protocol codec: <pre><code> let conn = await switch2.dial(switch1.peerInfo.peerId, switch1.peerInfo.addrs, PingCodec)\n</code></pre> We now have a <code>Ping</code> connection setup between the second and the first switch, we can use it to actually ping the node: <pre><code> # ping the other node and echo the ping duration\necho \"ping: \", await pingProtocol.ping(conn)\n# We must close the connection ourselves when we're done with it\nawait conn.close()\n</code></pre> And that's it! Just a little bit of cleanup: shutting down the switches, waiting for them to stop, and we'll call our <code>main</code> procedure: <pre><code> await allFutures(switch1.stop(), switch2.stop()) # close connections and shutdown all transports\nwaitFor(main())\n</code></pre> You can now run this program using <code>nim c -r part1.nim</code>, and you should see the dialing sequence, ending with a ping output.</p> <p>In the next tutorial, we'll look at how to create our own custom protocol.</p>"},{"location":"tutorial_2_customproto/","title":"Custom protocol in libp2p","text":"<p>In the previous tutorial, we've looked at how to create a simple ping program using the <code>nim-libp2p</code>.</p> <p>We'll now look at how to create a custom protocol inside the libp2p</p> <p>Let's create a <code>part2.nim</code>, and import our dependencies: <pre><code>import chronos\nimport stew/byteutils\nimport libp2p\n</code></pre> This is similar to the first tutorial, except we don't need to import the <code>Ping</code> protocol.</p> <p>Next, we'll declare our custom protocol <pre><code>const TestCodec = \"/test/proto/1.0.0\"\ntype TestProto = ref object of LPProtocol\n</code></pre> We've set a protocol ID, and created a custom <code>LPProtocol</code>. In a more complex protocol, we could use this structure to store interesting variables.</p> <p>A protocol generally has two part: and handling/server part, and a dialing/client part. Theses two parts can be identical, but in our trivial protocol, the server will wait for a message from the client, and the client will send a message, so we have to handle the two cases separately.</p> <p>Let's start with the server part: <pre><code>proc new(T: typedesc[TestProto]): T =\n# every incoming connections will in be handled in this closure\nproc handle(conn: Connection, proto: string) {.async, gcsafe.} =\n# Read up to 1024 bytes from this connection, and transform them into\n# a string\necho \"Got from remote - \", string.fromBytes(await conn.readLp(1024))\n# We must close the connections ourselves when we're done with it\nawait conn.close()\nreturn T.new(codecs = @[TestCodec], handler = handle)\n</code></pre> This is a constructor for our <code>TestProto</code>, that will specify our <code>codecs</code> and a <code>handler</code>, which will be called for each incoming peer asking for this protocol. In our handle, we simply read a message from the connection and <code>echo</code> it.</p> <p>We can now create our client part: <pre><code>proc hello(p: TestProto, conn: Connection) {.async.} =\nawait conn.writeLp(\"Hello p2p!\")\n</code></pre> Again, pretty straight-forward, we just send a message on the connection.</p> <p>We can now create our main procedure: <pre><code>proc main() {.async, gcsafe.} =\nlet\nrng = newRng()\ntestProto = TestProto.new()\nswitch1 = newStandardSwitch(rng=rng)\nswitch2 = newStandardSwitch(rng=rng)\nswitch1.mount(testProto)\nawait switch1.start()\nawait switch2.start()\nlet conn = await switch2.dial(switch1.peerInfo.peerId, switch1.peerInfo.addrs, TestCodec)\nawait testProto.hello(conn)\n# We must close the connection ourselves when we're done with it\nawait conn.close()\nawait allFutures(switch1.stop(), switch2.stop()) # close connections and shutdown all transports\n</code></pre> This is very similar to the first tutorial's <code>main</code>, the only noteworthy difference is that we use <code>newStandardSwitch</code>, which is similar to the <code>createSwitch</code> of the first tutorial, but is bundled directly in libp2p</p> <p>We can now wrap our program by calling our main proc: <pre><code>waitFor(main())\n</code></pre> And that's it! In the next tutorial, we'll create a more complex protocol using Protobuf.</p>"},{"location":"tutorial_3_protobuf/","title":"Protobuf usage","text":"<p>In the previous tutorial, we created a simple \"ping\" protocol. Most real protocol want their messages to be structured and extensible, which is why most real protocols use protobuf to define their message structures.</p> <p>Here, we'll create a slightly more complex protocol, which parses & generate protobuf messages. Let's start by importing our dependencies, as usual: <pre><code>import chronos\nimport stew/results # for Opt[T]\nimport libp2p\n</code></pre></p>"},{"location":"tutorial_3_protobuf/#protobuf-encoding-decoding","title":"Protobuf encoding & decoding","text":"<p>This will be the structure of our messages: <pre><code>message MetricList {\nmessage Metric {\nstring name = 1;\nfloat value = 2;\n}\nrepeated Metric metrics = 2;\n}\n</code></pre> We'll create our protobuf types, encoders & decoders, according to this format. To create the encoders & decoders, we are going to use minprotobuf (included in libp2p).</p> <p>While more modern technics (such as nim-protobuf-serialization) exists, minprotobuf is currently the recommended method to handle protobuf, since it has been used in production extensively, and audited. <pre><code>type\nMetric = object\nname: string\nvalue: float\nMetricList = object\nmetrics: seq[Metric]\n{.push raises: [].}\nproc encode(m: Metric): ProtoBuffer =\n result = initProtoBuffer()\n result.write(1, m.name)\n result.write(2, m.value)\n result.finish()\nproc decode(_: type Metric, buf: seq[byte]): Result[Metric, ProtoError] =\n var res: Metric\nlet pb = initProtoBuffer(buf)\n # \"getField\" will return a Result[bool, ProtoError].\n # The Result will hold an error if the protobuf is invalid.\n # The Result will hold \"false\" if the field is missing\n#\n # We are just checking the error, and ignoring whether the value\n# is present or not (default values are valid).\n discard ? pb.getField(1, res.name)\n discard ? pb.getField(2, res.value)\n ok(res)\nproc encode(m: MetricList): ProtoBuffer =\n result = initProtoBuffer()\n for metric in m.metrics:\n result.write(1, metric.encode())\n result.finish()\nproc decode(_: type MetricList, buf: seq[byte]): Result[MetricList, ProtoError] =\n var\nres: MetricList\nmetrics: seq[seq[byte]]\n let pb = initProtoBuffer(buf)\n discard ? pb.getRepeatedField(1, metrics)\n for metric in metrics:\n res.metrics &= ? Metric.decode(metric)\n ok(res)\n</code></pre></p>"},{"location":"tutorial_3_protobuf/#results-instead-of-exceptions","title":"Results instead of exceptions","text":"<p>As you can see, this part of the program also uses Results instead of exceptions for error handling. We start by <code>{.push raises: [].}</code>, which will prevent every non-async function from raising exceptions.</p> <p>Then, we use nim-result to convey errors to function callers. A <code>Result[T, E]</code> will either hold a valid result of type T, or an error of type E.</p> <p>You can check if the call succeeded by using <code>res.isOk</code>, and then get the value using <code>res.value</code> or the error by using <code>res.error</code>.</p> <p>Another useful tool is <code>?</code>, which will unpack a Result if it succeeded, or if it failed, exit the current procedure returning the error.</p> <p>nim-result is packed with other functionalities that you'll find in the nim-result repository.</p> <p>Results and exception are generally interchangeable, but have different semantics that you may or may not prefer.</p>"},{"location":"tutorial_3_protobuf/#creating-the-protocol","title":"Creating the protocol","text":"<p>We'll next create a protocol, like in the last tutorial, to request these metrics from our host <pre><code>type\nMetricCallback = proc: Future[MetricList] {.raises: [], gcsafe.}\nMetricProto = ref object of LPProtocol\nmetricGetter: MetricCallback\nproc new(_: typedesc[MetricProto], cb: MetricCallback): MetricProto =\nvar res: MetricProto\nproc handle(conn: Connection, proto: string) {.async, gcsafe.} =\nlet\nmetrics = await res.metricGetter()\nasProtobuf = metrics.encode()\nawait conn.writeLp(asProtobuf.buffer)\nawait conn.close()\nres = MetricProto.new(@[\"/metric-getter/1.0.0\"], handle)\nres.metricGetter = cb\nreturn res\nproc fetch(p: MetricProto, conn: Connection): Future[MetricList] {.async.} =\nlet protobuf = await conn.readLp(2048)\n# tryGet will raise an exception if the Result contains an error.\n# It's useful to bridge between exception-world and result-world\nreturn MetricList.decode(protobuf).tryGet()\n</code></pre> We can now create our main procedure: <pre><code>proc main() {.async, gcsafe.} =\nlet rng = newRng()\nproc randomMetricGenerator: Future[MetricList] {.async.} =\nlet metricCount = rng[].generate(uint32) mod 16\nfor i in 0 ..< metricCount + 1:\nresult.metrics.add(Metric(\nname: \"metric_\" & $i,\nvalue: float(rng[].generate(uint16)) / 1000.0\n))\nreturn result\nlet\nmetricProto1 = MetricProto.new(randomMetricGenerator)\nmetricProto2 = MetricProto.new(randomMetricGenerator)\nswitch1 = newStandardSwitch(rng=rng)\nswitch2 = newStandardSwitch(rng=rng)\nswitch1.mount(metricProto1)\nawait switch1.start()\nawait switch2.start()\nlet\nconn = await switch2.dial(switch1.peerInfo.peerId, switch1.peerInfo.addrs, metricProto2.codecs)\nmetrics = await metricProto2.fetch(conn)\nawait conn.close()\nfor metric in metrics.metrics:\necho metric.name, \" = \", metric.value\nawait allFutures(switch1.stop(), switch2.stop()) # close connections and shutdown all transports\nwaitFor(main())\n</code></pre> If you run this program, you should see random metrics being sent from the switch1 to the switch2.</p>"},{"location":"tutorial_4_gossipsub/","title":"GossipSub","text":"<p>In this tutorial, we'll build a simple GossipSub network to broadcast the metrics we built in the previous tutorial.</p> <p>GossipSub is used to broadcast some messages in a network, and allows to balance between latency, bandwidth usage, privacy and attack resistance.</p> <p>You'll find a good explanation on how GossipSub works here. There are a lot of parameters you can tweak to adjust how GossipSub behaves but here we'll use the sane defaults shipped with libp2p.</p> <p>We'll start by creating our metric structure like previously <pre><code>import chronos\nimport stew/results\nimport libp2p\nimport libp2p/protocols/pubsub/rpc/messages\ntype\nMetric = object\nname: string\nvalue: float\nMetricList = object\nhostname: string\nmetrics: seq[Metric]\n{.push raises: [].}\nproc encode(m: Metric): ProtoBuffer =\n result = initProtoBuffer()\n result.write(1, m.name)\n result.write(2, m.value)\n result.finish()\nproc decode(_: type Metric, buf: seq[byte]): Result[Metric, ProtoError] =\n var res: Metric\nlet pb = initProtoBuffer(buf)\n discard ? pb.getField(1, res.name)\n discard ? pb.getField(2, res.value)\n ok(res)\nproc encode(m: MetricList): ProtoBuffer =\n result = initProtoBuffer()\n for metric in m.metrics:\n result.write(1, metric.encode())\n result.write(2, m.hostname)\n result.finish()\nproc decode(_: type MetricList, buf: seq[byte]): Result[MetricList, ProtoError] =\n var\nres: MetricList\nmetrics: seq[seq[byte]]\n let pb = initProtoBuffer(buf)\n discard ? pb.getRepeatedField(1, metrics)\n for metric in metrics:\n res.metrics &= ? Metric.decode(metric)\n ? pb.getRequiredField(2, res.hostname)\n ok(res)\n</code></pre> This is exactly like the previous structure, except that we added a <code>hostname</code> to distinguish where the metric is coming from.</p> <p>Now we'll create a small GossipSub network to broadcast the metrics, and collect them on one of the node. <pre><code>type Node = tuple[switch: Switch, gossip: GossipSub, hostname: string]\nproc oneNode(node: Node, rng: ref HmacDrbgContext) {.async.} =\n# This procedure will handle one of the node of the network\nnode.gossip.addValidator([\"metrics\"],\nproc(topic: string, message: Message): Future[ValidationResult] {.async.} =\nlet decoded = MetricList.decode(message.data)\nif decoded.isErr: return ValidationResult.Reject\nreturn ValidationResult.Accept\n)\n# This \"validator\" will attach to the `metrics` topic and make sure\n# that every message in this topic is valid. This allows us to stop\n# propagation of invalid messages quickly in the network, and punish\n# peers sending them.\n# `John` will be responsible to log the metrics, the rest of the nodes\n# will just forward them in the network\nif node.hostname == \"John\":\nnode.gossip.subscribe(\"metrics\",\nproc (topic: string, data: seq[byte]) {.async.} =\necho MetricList.decode(data).tryGet()\n)\nelse:\nnode.gossip.subscribe(\"metrics\", nil)\n# Create random metrics 10 times and broadcast them\nfor _ in 0..<10:\nawait sleepAsync(500.milliseconds)\nvar metricList = MetricList(hostname: node.hostname)\nlet metricCount = rng[].generate(uint32) mod 4\nfor i in 0 ..< metricCount + 1:\nmetricList.metrics.add(Metric(\nname: \"metric_\" & $i,\nvalue: float(rng[].generate(uint16)) / 1000.0\n))\ndiscard await node.gossip.publish(\"metrics\", encode(metricList).buffer)\nawait node.switch.stop()\n</code></pre> For our main procedure, we'll create a few nodes, and connect them together. Note that they are not all interconnected, but GossipSub will take care of broadcasting to the full network nonetheless. <pre><code>proc main {.async.} =\nlet rng = newRng()\nvar nodes: seq[Node]\nfor hostname in [\"John\", \"Walter\", \"David\", \"Thuy\", \"Amy\"]:\nlet\nswitch = newStandardSwitch(rng=rng)\ngossip = GossipSub.init(switch = switch, triggerSelf = true)\nswitch.mount(gossip)\nawait switch.start()\nnodes.add((switch, gossip, hostname))\nfor index, node in nodes:\n# Connect to a few neighbors\nfor otherNodeIdx in index - 1 .. index + 2:\nif otherNodeIdx notin 0 ..< nodes.len or otherNodeIdx == index: continue\nlet otherNode = nodes[otherNodeIdx]\nawait node.switch.connect(\notherNode.switch.peerInfo.peerId,\notherNode.switch.peerInfo.addrs)\nvar allFuts: seq[Future[void]]\nfor node in nodes:\nallFuts.add(oneNode(node, rng))\nawait allFutures(allFuts)\nwaitFor(main())\n</code></pre> If you run this program, you should see something like: <pre><code>(hostname: \"John\", metrics: @[(name: \"metric_0\", value: 42.097), (name: \"metric_1\", value: 50.99), (name: \"metric_2\", value: 47.86), (name: \"metric_3\", value: 5.368)])\n(hostname: \"Walter\", metrics: @[(name: \"metric_0\", value: 39.452), (name: \"metric_1\", value: 15.606), (name: \"metric_2\", value: 14.059), (name: \"metric_3\", value: 6.68)])\n(hostname: \"David\", metrics: @[(name: \"metric_0\", value: 9.82), (name: \"metric_1\", value: 2.862), (name: \"metric_2\", value: 15.514)])\n(hostname: \"Thuy\", metrics: @[(name: \"metric_0\", value: 59.038)])\n(hostname: \"Amy\", metrics: @[(name: \"metric_0\", value: 55.616), (name: \"metric_1\", value: 23.52), (name: \"metric_2\", value: 59.081), (name: \"metric_3\", value: 2.516)])\n</code></pre></p> <p>This is John receiving & logging everyone's metrics.</p>"},{"location":"tutorial_4_gossipsub/#going-further","title":"Going further","text":"<p>Building efficient & safe GossipSub networks is a tricky subject. By tweaking the gossip params and topic params, you can achieve very different properties.</p> <p>Also see reports for GossipSub v1.1</p> <p>If you are interested in broadcasting for your application, you may want to use Waku, which builds on top of GossipSub, and adds features such as history, spam protection, and light node friendliness.</p>"},{"location":"tutorial_5_discovery/","title":"Discovery Manager","text":"<p>In the previous tutorial, we built a custom protocol using protobuf and spread informations (some metrics) on the network using gossipsub. For this tutorial, on the other hand, we'll go back on a simple example we'll try to discover a specific peers to greet on the network.</p> <p>First, as usual, we import the dependencies: <pre><code>import sequtils\nimport chronos\nimport stew/byteutils\nimport libp2p\nimport libp2p/protocols/rendezvous\nimport libp2p/discovery/rendezvousinterface\nimport libp2p/discovery/discoverymngr\n</code></pre> We'll not use newStandardSwitch this time as we need the discovery protocol RendezVous to be mounted on the switch using withRendezVous.</p> <p>Note that other discovery methods such as Kademlia or discv5 exist. <pre><code>proc createSwitch(rdv: RendezVous = RendezVous.new()): Switch =\nSwitchBuilder.new()\n.withRng(newRng())\n.withAddresses(@[ MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet() ])\n.withTcpTransport()\n.withYamux()\n.withNoise()\n.withRendezVous(rdv)\n.build()\n# Create a really simple protocol to log one message received then close the stream\nconst DumbCodec = \"/dumb/proto/1.0.0\"\ntype DumbProto = ref object of LPProtocol\nproc new(T: typedesc[DumbProto], nodeNumber: int): T =\nproc handle(conn: Connection, proto: string) {.async, gcsafe.} =\necho \"Node\", nodeNumber, \" received: \", string.fromBytes(await conn.readLp(1024))\nawait conn.close()\nreturn T.new(codecs = @[DumbCodec], handler = handle)\n</code></pre></p>"},{"location":"tutorial_5_discovery/#bootnodes","title":"Bootnodes","text":"<p>The first time a p2p program is ran, he needs to know how to join its network. This is generally done by hard-coding a list of stable nodes in the binary, called \"bootnodes\". These bootnodes are a critical part of a p2p network, since they are used by every new user to onboard the network.</p> <p>By using libp2p, we can use any node supporting our discovery protocol (rendezvous in this case) as a bootnode. For this example, we'll create a bootnode, and then every peer will advertise itself on the bootnode, and use it to find other peers <pre><code>proc main() {.async, gcsafe.} =\nlet bootNode = createSwitch()\nawait bootNode.start()\n# Create 5 nodes in the network\nvar\nswitches: seq[Switch] = @[]\ndiscManagers: seq[DiscoveryManager] = @[]\nfor i in 0..5:\nlet rdv = RendezVous.new()\n# Create a remote future to await at the end of the program\nlet switch = createSwitch(rdv)\nswitch.mount(DumbProto.new(i))\nswitches.add(switch)\n# A discovery manager is a simple tool, you can set it up by adding discovery\n# interfaces (such as RendezVousInterface) then you can use it to advertise\n# something on the network or to request something from it.\nlet dm = DiscoveryManager()\n# A RendezVousInterface is a RendezVous protocol wrapped to be usable by the\n# DiscoveryManager.\ndm.add(RendezVousInterface.new(rdv))\ndiscManagers.add(dm)\n# We can now start the switch and connect to the bootnode\nawait switch.start()\nawait switch.connect(bootNode.peerInfo.peerId, bootNode.peerInfo.addrs)\n# Each nodes of the network will advertise on some topics (EvenGang or OddClub)\ndm.advertise(RdvNamespace(if i mod 2 == 0: \"EvenGang\" else: \"OddClub\"))\n</code></pre> We can now create the newcomer. This peer will connect to the boot node, and use it to discover peers & greet them.</p> <pre><code> let\nrdv = RendezVous.new()\nnewcomer = createSwitch(rdv)\ndm = DiscoveryManager()\nawait newcomer.start()\nawait newcomer.connect(bootNode.peerInfo.peerId, bootNode.peerInfo.addrs)\ndm.add(RendezVousInterface.new(rdv, ttr = 250.milliseconds))\n# Use the discovery manager to find peers on the OddClub topic to greet them\nlet queryOddClub = dm.request(RdvNamespace(\"OddClub\"))\nfor _ in 0..2:\nlet\n# getPeer give you a PeerAttribute containing informations about the peer.\nres = await queryOddClub.getPeer()\n# Here we will use the PeerId and the MultiAddress to greet him\nconn = await newcomer.dial(res[PeerId], res.getAll(MultiAddress), DumbCodec)\nawait conn.writeLp(\"Odd Club suuuucks! Even Gang is better!\")\n# Uh-oh!\nawait conn.close()\n# Wait for the peer to close the stream\nawait conn.join()\n# Queries will run in a loop, so we must stop them when we are done\nqueryOddClub.stop()\n# Maybe it was because he wanted to join the EvenGang\nlet queryEvenGang = dm.request(RdvNamespace(\"EvenGang\"))\nfor _ in 0..2:\nlet\nres = await queryEvenGang.getPeer()\nconn = await newcomer.dial(res[PeerId], res.getAll(MultiAddress), DumbCodec)\nawait conn.writeLp(\"Even Gang is sooo laaame! Odd Club rocks!\")\n# Or maybe not...\nawait conn.close()\nawait conn.join()\nqueryEvenGang.stop()\n# What can I say, some people just want to watch the world burn... Anyway\n# Stop all the discovery managers\nfor d in discManagers:\nd.stop()\ndm.stop()\n# Stop all the switches\nawait allFutures(switches.mapIt(it.stop()))\nawait allFutures(bootNode.stop(), newcomer.stop())\nwaitFor(main())\n</code></pre>"},{"location":"tutorial_6_game/","title":"Tron example","text":"<p>In this tutorial, we will create a video game based on libp2p, using all of the features we talked about in the last tutorials.</p> <p>We will: - Discover peers using the Discovery Manager - Use GossipSub to find a play mate - Create a custom protocol to play with him</p> <p>While this may look like a daunting project, it's less than 150 lines of code.</p> <p>The game will be a simple Tron. We will use nico as a game engine. (you need to run <code>nimble install nico</code> to have it available)</p> <p></p> <p>We will start by importing our dependencies and creating our types <pre><code>import os\nimport nico, chronos, stew/byteutils, stew/endians2\nimport libp2p\nimport libp2p/protocols/rendezvous\nimport libp2p/discovery/rendezvousinterface\nimport libp2p/discovery/discoverymngr\nconst\ndirections = @[(K_UP, 0, -1), (K_LEFT, -1, 0), (K_DOWN, 0, 1), (K_RIGHT, 1, 0)]\nmapSize = 32\ntickPeriod = 0.2\ntype\nPlayer = ref object\nx, y: int\ncurrentDir, nextDir: int\nlost: bool\ncolor: int\nGame = ref object\ngameMap: array[mapSize * mapSize, int]\ntickTime: float\nlocalPlayer, remotePlayer: Player\npeerFound: Future[Connection]\nhasCandidate: bool\ntickFinished: Future[int]\nGameProto = ref object of LPProtocol\nproc new(_: type[Game]): Game =\n# Default state of a game\nresult = Game(\ntickTime: -3.0, # 3 seconds of \"warm-up\" time\nlocalPlayer: Player(x: 4, y: 16, currentDir: 3, nextDir: 3, color: 8),\nremotePlayer: Player(x: 27, y: 16, currentDir: 1, nextDir: 1, color: 12),\npeerFound: newFuture[Connection]()\n)\nfor pos in 0 .. result.gameMap.high:\nif pos mod mapSize in [0, mapSize - 1] or pos div mapSize in [0, mapSize - 1]:\nresult.gameMap[pos] = 7\n</code></pre></p>"},{"location":"tutorial_6_game/#game-logic","title":"Game Logic","text":"<p>The networking during the game will work like this:</p> <ul> <li>Each player will have <code>tickPeriod</code> (0.1) seconds to choose a direction that he wants to go to (default to current direction)</li> <li>After <code>tickPeriod</code>, we will send our choosen direction to the peer, and wait for his direction</li> <li>Once we have both direction, we will \"tick\" the game, and restart the loop, as long as both player are alive.</li> </ul> <p>This is a very simplistic scheme, but creating proper networking for video games is an art</p> <p>The main drawback of this scheme is that the more ping you have with the peer, the slower the game will run. Or invertedly, the less ping you have, the faster it runs! <pre><code>proc update(g: Game, dt: float32) =\n# Will be called at each frame of the game.\n#\n# Because both Nico and Chronos have a main loop,\n# they must share the control of the main thread.\n# This is a hacky way to make this happen\nwaitFor(sleepAsync(1.milliseconds))\n# Don't do anything if we are still waiting for an opponent\nif not(g.peerFound.finished()) or isNil(g.tickFinished): return\ng.tickTime += dt\n# Update the wanted direction, making sure we can't go backward\nfor i in 0 .. directions.high:\nif i != (g.localPlayer.currentDir + 2 mod 4) and keyp(directions[i][0]):\ng.localPlayer.nextDir = i\nif g.tickTime > tickPeriod and not g.tickFinished.finished():\n# We choosen our next direction, let the networking know\ng.localPlayer.currentDir = g.localPlayer.nextDir\ng.tickFinished.complete(g.localPlayer.currentDir)\nproc tick(g: Game, p: Player) =\n# Move player and check if he lost\np.x += directions[p.currentDir][1]\np.y += directions[p.currentDir][2]\nif g.gameMap[p.y * mapSize + p.x] != 0: p.lost = true\ng.gameMap[p.y * mapSize + p.x] = p.color\nproc mainLoop(g: Game, peer: Connection) {.async.} =\nwhile not (g.localPlayer.lost or g.remotePlayer.lost):\nif g.tickTime > 0.0:\ng.tickTime = 0\ng.tickFinished = newFuture[int]()\n# Wait for a choosen direction\nlet dir = await g.tickFinished\n# Send it\nawait peer.writeLp(toBytes(uint32(dir)))\n# Get the one from the peer\ng.remotePlayer.currentDir = int uint32.fromBytes(await peer.readLp(8))\n# Tick the players & restart\ng.tick(g.remotePlayer)\ng.tick(g.localPlayer)\n</code></pre> We'll draw the map & put some texts when necessary: <pre><code>proc draw(g: Game) =\nfor pos, color in g.gameMap:\nsetColor(color)\nboxFill(pos mod 32 * 4, pos div 32 * 4, 4, 4)\nlet text = if not(g.peerFound.finished()): \"Matchmaking..\"\nelif g.tickTime < -1.5: \"Welcome to Etron\"\nelif g.tickTime < 0.0: \"- \" & $(int(abs(g.tickTime) / 0.5) + 1) & \" -\"\nelif g.remotePlayer.lost and g.localPlayer.lost: \"DEUCE\"\nelif g.localPlayer.lost: \"YOU LOOSE\"\nelif g.remotePlayer.lost: \"YOU WON\"\nelse: \"\"\nprintc(text, screenWidth div 2, screenHeight div 2)\n</code></pre></p>"},{"location":"tutorial_6_game/#matchmaking","title":"Matchmaking","text":"<p>To find an opponent, we will broadcast our address on a GossipSub topic, and wait for someone to connect to us. We will also listen to that topic, and connect to anyone broadcasting his address.</p> <p>If we are looking for a game, we'll send <code>ok</code> to let the peer know that we are available, check that he is also available, and launch the game. <pre><code>proc new(T: typedesc[GameProto], g: Game): T =\nproc handle(conn: Connection, proto: string) {.async, gcsafe.} =\ndefer: await conn.closeWithEof()\nif g.peerFound.finished or g.hasCandidate:\nawait conn.close()\nreturn\ng.hasCandidate = true\nawait conn.writeLp(\"ok\")\nif \"ok\" != string.fromBytes(await conn.readLp(1024)):\ng.hasCandidate = false\nreturn\ng.peerFound.complete(conn)\n# The handler of a protocol must wait for the stream to\n# be finished before returning\nawait conn.join()\nreturn T.new(codecs = @[\"/tron/1.0.0\"], handler = handle)\nproc networking(g: Game) {.async.} =\n# Create our switch, similar to the GossipSub example and\n# the Discovery examples combined\nlet\nrdv = RendezVous.new()\nswitch = SwitchBuilder.new()\n.withRng(newRng())\n.withAddresses(@[ MultiAddress.init(\"/ip4/0.0.0.0/tcp/0\").tryGet() ])\n.withTcpTransport()\n.withYamux()\n.withNoise()\n.withRendezVous(rdv)\n.build()\ndm = DiscoveryManager()\ngameProto = GameProto.new(g)\ngossip = GossipSub.init(\nswitch = switch,\ntriggerSelf = false)\ndm.add(RendezVousInterface.new(rdv))\nswitch.mount(gossip)\nswitch.mount(gameProto)\ngossip.subscribe(\n\"/tron/matchmaking\",\nproc (topic: string, data: seq[byte]) {.async.} =\n# If we are still looking for an opponent,\n# try to match anyone broadcasting it's address\nif g.peerFound.finished or g.hasCandidate: return\ng.hasCandidate = true\ntry:\nlet\n(peerId, multiAddress) = parseFullAddress(data).tryGet()\nstream = await switch.dial(peerId, @[multiAddress], gameProto.codec)\nawait stream.writeLp(\"ok\")\nif (await stream.readLp(10)) != \"ok\".toBytes:\ng.hasCandidate = false\nreturn\ng.peerFound.complete(stream)\n# We are \"player 2\"\nswap(g.localPlayer, g.remotePlayer)\nexcept CatchableError as exc:\ndiscard\n)\nawait switch.start()\ndefer: await switch.stop()\n# As explained in the last tutorial, we need a bootnode to be able\n# to find peers. We could use any libp2p running rendezvous (or any\n# node running tron). We will take it's MultiAddress from the command\n# line parameters\nif paramCount() > 0:\nlet (peerId, multiAddress) = paramStr(1).parseFullAddress().tryGet()\nawait switch.connect(peerId, @[multiAddress])\nelse:\necho \"No bootnode provided, listening on: \", switch.peerInfo.fullAddrs.tryGet()\n# Discover peers from the bootnode, and connect to them\ndm.advertise(RdvNamespace(\"tron\"))\nlet discoveryQuery = dm.request(RdvNamespace(\"tron\"))\ndiscoveryQuery.forEach:\ntry:\nawait switch.connect(peer[PeerId], peer.getAll(MultiAddress))\nexcept CatchableError as exc:\necho \"Failed to dial a peer: \", exc.msg\n# We will try to publish our address multiple times, in case\n# it takes time to establish connections with other GossipSub peers\nvar published = false\nwhile not published:\nawait sleepAsync(500.milliseconds)\nfor fullAddr in switch.peerInfo.fullAddrs.tryGet():\nif (await gossip.publish(\"/tron/matchmaking\", fullAddr.bytes)) == 0:\npublished = false\nbreak\npublished = true\ndiscoveryQuery.stop()\n# We now wait for someone to connect to us (or for us to connect to someone)\nlet peerConn = await g.peerFound\ndefer: await peerConn.closeWithEof()\nawait g.mainLoop(peerConn)\nlet\ngame = Game.new()\nnetFut = networking(game)\nnico.init(\"Status\", \"Tron\")\nnico.createWindow(\"Tron\", mapSize * 4, mapSize * 4, 4, false)\nnico.run(proc = discard, proc(dt: float32) = game.update(dt), proc = game.draw())\nwaitFor(netFut.cancelAndWait())\n</code></pre> And that's it! If you want to run this code locally, the simplest way is to use the first node as a boot node for the second one. But you can also use any rendezvous node</p>"},{"location":"go-daemon/daemonapi/","title":"Table of Contents","text":"<ul> <li>Introduction</li> <li>Installation</li> <li>Usage</li> <li>Example</li> <li>Getting Started</li> </ul>"},{"location":"go-daemon/daemonapi/#introduction","title":"Introduction","text":"<p>This is a libp2p-backed daemon wrapping the functionalities of go-libp2p for use in Nim. For more information about the go daemon, check out this repository.</p>"},{"location":"go-daemon/daemonapi/#installation","title":"Installation","text":"<pre><code># clone and install dependencies\ngit clone https://github.com/status-im/nim-libp2p\ncd nim-libp2p\nnimble install\n\n# perform unit tests\nnimble test\n# update the git submodule to install the go daemon \ngit submodule update --init --recursive\ngo version\ngit clone https://github.com/libp2p/go-libp2p-daemon\ncd go-libp2p-daemon\ngit checkout v0.0.1\ngo install ./...\ncd ..\n</code></pre>"},{"location":"go-daemon/daemonapi/#usage","title":"Usage","text":""},{"location":"go-daemon/daemonapi/#example","title":"Example","text":"<p>Examples can be found in the examples folder</p>"},{"location":"go-daemon/daemonapi/#getting-started","title":"Getting Started","text":"<p>Try out the chat example. Full code can be found here:</p> <pre><code>nim c -r --threads:on examples/directchat.nim\n</code></pre> <p>This will output a peer ID such as <code>QmbmHfVvouKammmQDJck4hz33WvVktNEe7pasxz2HgseRu</code> which you can use in another instance to connect to it.</p> <pre><code>./examples/directchat\n/connect QmbmHfVvouKammmQDJck4hz33WvVktNEe7pasxz2HgseRu\n</code></pre> <p>You can now chat between the instances!</p> <p></p>"}]} |