Vac deep research experiments https://status-im.github.io/nim-libp2p/docs/
Go to file
Dmitriy Ryajov 0959877b29
Connection limits (#384)
* master merge

* wip

* avoid deadlocks

* tcp limits

* expose client field in chronosstream

* limit incoming connections

* update with new listen api

* fix release

* don't override peerinfo in connection

* rework transport with accept

* use semaphore to track resource ussage

* rework with new transport accept api

* move events to conn manager (#373)

* use semaphore to track resource ussage

* merge master

* expose api to acquire conn slots

* don't fail expensive metrics

* allow tracking and updating connections

* set global connection limits to 80

* add per peer connection limits

* make sure conn is closed if tracking failed

* more descriptive naming for handle

* rework with new transport accept api

* add `getStream` hide `selectConn`

* add TransportClosedError

* make nil explicit

* don't make unnecessary copies of message

* logging

* error handling

* cleanup semaphore

* track connections properly

* throw `TooManyConnections` when tracking outgoing

* use proper exception and handle conventions

* check onCloseHandle for nil

* revert internalConnect changes

* adding upgraded flag

* await stream before closing

* simplify tracking

* wip

* logging

* split connection limits into incoming and outgoing

* further streamline connection limits split counts

* don't use closeWithEOF

* move peer and conn event triggers from switch

* wip

* wip

* wip

* merge master

* handle nil connections properly

* add clarifying comment

* don't raise exc on nil

* no finally

* add proper min/max connections logic

* rebase master

* merge master

* master merge

* remove request timeout

should be addressed in separate PR

* merge master

* share semaphore when in/out limits arent enforced

* merge master

* use import

* pass semaphore to trackConn

* don't close last conn

* use storeConn

* merge master

* use storeConn
2021-01-20 22:00:24 -06:00
.github/workflows Enable gh actions to run on multiple platforms (#479) 2020-12-22 18:12:02 -06:00
docs replace nim-beacon-chain URLs with nimbus-eth2 URLs (#396) 2020-10-09 18:02:03 +02:00
examples Gossip one one (#240) 2020-09-21 11:16:29 +02:00
libp2p Connection limits (#384) 2021-01-20 22:00:24 -06:00
scripts Enable gh actions to run on multiple platforms (#479) 2020-12-22 18:12:02 -06:00
tests Connection limits (#384) 2021-01-20 22:00:24 -06:00
tools Add `libp2p_dump` and `libp2p_dump_dir` compiler time options. (#359) 2020-09-15 11:16:43 +02:00
.appveyor.yml Gossip one one (#240) 2020-09-21 11:16:29 +02:00
.gitignore Gossip one one (#240) 2020-09-21 11:16:29 +02:00
LICENSE Initial commit 2018-11-19 03:58:31 +02:00
LICENSE-APACHEv2 Initialize travis and licenses. 2018-11-19 04:04:47 +02:00
LICENSE-MIT Initialize travis and licenses. 2018-11-19 04:04:47 +02:00
README.md add metrics into chronosstream to identify peers agents (#458) 2021-01-08 14:21:24 +09:00
codecov.yml Gossipsub refactor (#490) 2020-12-19 15:43:32 +01:00
libp2p.nim Fixes and tweaks related to the beacon node integration 2019-12-08 23:58:43 +02:00
libp2p.nimble Gossipsub refactor (#490) 2020-12-19 15:43:32 +01:00

README.md

libp2p hex logo

The Nim implementation of the libp2p Networking Stack.

Introduction

An implementation of libp2p in Nim. Also provides a Nim wrapper of the Libp2p Go daemon.

Project Status

The current native Nim libp2p implementation support is experimental and shouldn't be relied on for production use. It is under active development and contributions are highly welcomed. :)

Check our examples folder to get started!

Table of Contents

Background

libp2p is a networking stack and library modularized out of The IPFS Project, and bundled separately for other tools to use.

libp2p is the product of a long and arduous quest of understanding; a deep dive into the internet's network stack and the peer-to-peer protocols from the past. Building large scale peer-to-peer systems has been complex and difficult in the last 15 years and libp2p is a way to fix that. It is a "network stack", a suite of networking protocols that cleanly separates concerns and enables sophisticated applications to only use the protocols they absolutely need, without giving up interoperability and upgradeability.

libp2p grew out of IPFS, but it is built so that lots of people can use it, for lots of different projects.

  • Learn more about libp2p at libp2p.io and follow our evolving documentation efforts at docs.libp2p.io.
  • Here is an overview of libp2p and its implementations in other programming languages.

Install

nimble install libp2p

Prerequisite

Usage

API

The specification is available in the docs/api folder.

Getting Started

Please read the GETTING_STARTED.md guide.

Tutorials and Examples

Example code can be found in the examples folder.

Direct Chat Tutorial

  • Part I: Set up the main function and use multi-thread for processing IO.
  • Part II: Dial remote peer and allow customized user input commands.
  • Part III: Configure and establish a libp2p node.

Using the Go Daemon

Please find the installation and usage intructions in daemonapi.md.

Examples can be found in the examples/go-daemon folder;

Development

Clone and Install dependencies:

git clone https://github.com/status-im/nim-libp2p
cd nim-libp2p
nimble install

Tests

Prerequisite

Run unit tests

# run all the unit tests
nimble test

Packages

List of packages currently in existence for nim-libp2p:

Libp2p

Transports

Secure Channels

Stream Multiplexers

Utilities

Data Types

Pubsub

Packages that exist in the original libp2p specs and are under active development:

  • libp2p-daemon
  • libp2p-webrtc-direct
  • libp2p-webrtc-star
  • libp2p-websockets
  • libp2p-spdy
  • libp2p-bootstrap
  • libp2p-kad-dht
  • libp2p-mdns
  • libp2p-webrtc-star
  • libp2p-delegated-content-routing
  • libp2p-delegated-peer-routing
  • libp2p-nat-mgnr
  • libp2p-utils

** Note that the current stack reflects the minimal requirements for the upcoming Eth2 implementation.

Tips and tricks

enable expensive metrics:

nim c -d:libp2p_expensive_metrics some_file.nim

use identify metrics

nim c -d:libp2p_agents_metrics -d:KnownLibP2PAgents=nimbus,lighthouse,prysm,teku some_file.nim

specify gossipsub specific topics to measure

nim c -d:KnownLibP2PTopics=topic1,topic2,topic3 some_file.nim

Contribute

The libp2p implementation in Nim is a work in progress. We welcome contributors to help out! Specifically, you can:

  • Go through the modules and check out existing issues. This would be especially useful for modules in active development. Some knowledge of IPFS/libp2p may be required, as well as the infrastructure behind it.
  • Perform code reviews. Feel free to let us know if you found anything that can a) speed up the project development b) ensure better quality and c) reduce possible future bugs.
  • Add tests. Help nim-libp2p to be more robust by adding more tests to the tests folder.

Core Developers

@cheatfate, Dmitriy Ryajov, Giovanni Petrantoni, Zahary Karadjov

License

Licensed and distributed under either of

or

at your option. These files may not be copied, modified, or distributed except according to those terms.