Update documentation

README.md

@@ -3,111 +3,255 @@
 [![Build Status (Travis)](https://img.shields.io/travis/status-im/nim-eth-rpc/master.svg?label=Linux%20/%20macOS "Linux/macOS build status (Travis)")](https://travis-ci.org/status-im/nim-eth-rpc)
 [![Windows build status (Appveyor)](https://img.shields.io/appveyor/ci/jarradh/nim-eth-rpc/master.svg?label=Windows "Windows build status (Appveyor)")](https://ci.appveyor.com/project/jarradh/nim-eth-rpc)
 
-Json-Rpc is designed to provide an easier interface for working with remote procedure calls.
+Json-Rpc is a library designed to provide an easier interface for working with remote procedure calls.
 
 # Installation
 
 `git clone https://github.com/status-im/nim-eth-rpc`
 
 
-## Requirements
+### Requirements
 * Nim 17.3 and up
 
 
-# Usage
+# Introduction
 
-## Server
+Json-Rpc is a library for routing JSON 2.0 format remote procedure calls over different transports.
+It is designed to automatically generate marshalling and parameter checking code based on the RPC parameter types.
 
-Remote procedure calls are created using the `rpc` macro.
+## Routing
-This macro allows you to provide a list of native Nim type parameters and a return type, and will automatically handle all the marshalling to and from json for you, so you can concentrate on using native Nim types for your call.
 
-Here's a full example of a server with a single RPC.
+Remote procedure calls are created using the `rpc` macro on an instance of `RpcRouter`.
+
+`rpc` allows you to provide a list of native Nim type parameters and a return type, generates marshalling to and from json for you, so you can concentrate on using native Nim types for your call.
+
+Routing is then performed by the `route` procedure.
+
+When an error occurs, the `error` is populated, otherwise `result` will be populated.
+
+### Parameters
+
+`route`
+  `path`: The string to match for the `method`.
+  `body`: The parameters and code to execute for this call.
+
+### Example
+
+Here's a simple example:
 
 ```nim
 import rpcserver
 
-var srv = newRpcServer("")
+var router = newRpcRouter()
 
-# Create an RPC with a string an array parameter, that returns an int
+router.rpc:
-srv.rpc("myProc") do(input: int, data: array[0..3, int]) -> string:
+  result = %"Hello"
-  result = "Hello " & $input & " data: " & $data
-
-asyncCheck srv.serve()
-runForever()
 ```
 
-Parameter types are recursively traversed so you can use any custom types you wish, even nested types. Ref and object types are fully supported.
+As no return type was specified in this example, `result` defaults to the `JsonNode` type.
+A JSON string is returned by passing a string though the `%` operator, which converts simple types to `JsonNode`.
+
+The `body` parameters can be defined by using [do notation](https://nim-lang.org/docs/manual.html#procedures-do-notation).
+This allows full Nim types to be used as RPC parameters.
+
+Here we pass a string to an RPC and return a string.
+
+```nim
+router.rpc("hello") do(input: string) -> string:
+  result = "Hello " & input
+```
+
+Json-Rpc will recursively parse the Nim types in order to produce marshalling code.
+This marshalling code uses the types to check the incoming JSON fields to ensure they exist and are of the correct kind.
+
+The return type then performs the opposite process, converting Nim types to Json for transport.
+
+Here is a more complex parameter example:
 
 ```nim
 type
-  Details = ref object
+  HeaderKind = enum hkOne, hkTwo, hkThree
-    values: seq[byte]
   
-  Payload = object
+  Header = ref object
-    x, y: float
+    kind: HeaderKind
-    count: int
+    size: int64
-    details: Details
 
-  ResultData = object
+  DataBlob = object
-    data: array[10, byte]
+    items: seq[byte]
+    headers: array[3, Header]
 
-srv.rpc("getResults") do(payload: Payload) -> ResultData:
+  MyObject = object
-  # Here we can use Payload as expected, and `result` will be of type ResultData.
+    data: DataBlob
-  # Parameters and results are automatically converted to and from json
+    name: string
-  # and the call is intrinsically asynchronous.
 
+router.rpc("updateData") do(myObj: MyObject, newData: DataBlob) -> DataBlob:
+  result = myObj.data  
+  myObj.data = newData
 ```
 
-Behind the scenes, all RPC calls take a single json parameter that must be defined as a `JArray`.
+Behind the scenes, all RPC calls take a single json parameter `param` that must be of kind `JArray`.
 At runtime, the json is checked to ensure that it contains the correct number and type of your parameters to match the `rpc` definition.
-The `rpc` macro takes care of the boiler plate in marshalling to and from json.
 
-Compiling with `-d:nimDumpRpcs` will show the output code for the RPC call.
+Compiling with `-d:nimDumpRpcs` will show the output code for the RPC call. To see the output of the `async` generation, add `-d:nimDumpAsync`.
 
-The following RPC:
+## Format
 
-```nim
+The router expects either a string or `JsonNode` with the following structure:
-srv.rpc("myProc") do(input: string, data: array[0..3, int]):
-  result = %("Hello " & input & " data: " & $data)
-```
-Will get transformed into something like this:
 
-```nim
+```json
-proc myProc*(params: JsonNode): Future[JsonNode] {.async.} =
+{
-  params.kind.expect(JArray, "params")
+  "id": JInt,
-  if params.len != 2:
+  "jsonrpc": "2.0",
-    raise newException(ValueError, "Expected 2 Json parameter(s) but got " &
+  "method": JString,
-        $params.len)
+  "params": JArray
-  var input: string
+}
-  input = unpackArg(params.elems[0], "input", type(string))
-  var data: array[0 .. 3, int]
-  data = unpackArg(params.elems[1], "data", type(array[0 .. 3, int]))
-  result = %("Hello " & input & " data: " & $data)
 ```
 
-## Client
+Return values use the following node structure:
+
+```json
+{
+  "id": JInt,
+  "jsonrpc": "2.0",
+  "result": JsonNode,
+  "error": JsonNode
+}
+```
+
+## Performing a route
+
+To call the router, use the `route` procedure.
+
+There are three variants of `route`.
+
+Note that once invoked all RPC calls are error trapped and any exceptions raised are passed back with the error message encoded as a `JsonNode`.
+
+  `route`
+    `router: RpcRouter`: The router object that contains the RPCs.
+    `data: string`: A string ready to be processed into a `JsonNode`.
+    Returns
+    `Future[string]`: This will be the stringified JSON response, which can be the JSON RPC result or a JSON wrapped error.
+  
+  This `route` variant will handle all the conversion of `string` to `JsonNode` and check the format and type of the input data.
+
+  `route`
+    `router: RpcRouter`: The router object that contains the RPCs.
+    `node: JsonNode`: A pre-processed JsonNode that matches the expected format as defined above.
+    Returns
+    `Future[JsonNode]`: The JSON RPC result or a JSON wrapped error.
+  
+  This variant allows simplified processing if you already have a `JsonNode`. However if the required fields are not present within `node`, exceptions will be raised.
+
+  `tryRoute`
+    `router: RpcRouter`: The router object that contains the RPCs.
+    `node: JsonNode`: A pre-processed JsonNode that matches the expected format as defined above.
+    `fut: var Future[JsonNode]`: The JSON RPC result or a JSON wrapped error.
+    Returns
+    `bool`: Returns `true` if the `method` field provided in `node` matches an available route. Returns `false` when the `method` cannot be found, or if `method` or `params` field cannot be found within `node`.
+  
+  This `route` variant allows you to invoke a call if possible, without raising an exception.
+
+To see the result of a call, we need to provide Json in the expected format.
+Here's an example of how that looks by manually creating the JSON. Later we will see the helper utilities that make this easier.
+
+```nim
+let call = %*{
+  "id": %1,
+  "jsonrpc": %2.0,
+  "method": %"hello",
+  "params": %["Terry"]
+  }
+# route the call we defined earlier
+let localResult = waitFor router.route(call)
+
+echo localResult
+# We should see something like this
+#   {"jsonrpc":"2.0","id":1,"result":"Hello Terry","error":null}
+```
+
+# Server
+
+In order to make routing useful, RPCs must be invoked and transmitted over a transport.
+
+The `RpcServer` type is given as a simple inheritable wrapper/container that simplifies designing your own transport layers using the `router` field.
+
+## Server Transports
+
+Currently there are plans for the following transports to be implemented:
+
+* [x] Sockets
+* [ ] HTTP
+* [ ] IPC
+* [ ] Websockets
+
+Transport specific server need only call the `route` procedure using a string fetched from the transport in order to invoke the requested RPC.
+
+## Server example
+
+This example uses the socket transport defined in `socket.nim`.
+Once executed, the "hello" RPC will be available to a socket based client.
+
+```nim
+import rpcserver
+
+# Create a socket server for transport
+var srv = newRpcSocketServer("localhost", Port(8585))
+
+# srv.rpc is a shortcut for srv.router.rpc 
+srv.rpc("hello") do(input: string) -> string:
+  result = "Hello " & input
+
+srv.start()
+runForever()
+```
+
+# Client
+
+Json-Rpc also comes with a client implementation, built to provide a framework for transports to work with.
+
+To simplify demonstration, we will use the socket transport defined in `socketclient.nim`.
 
 Below is the most basic way to use a remote call on the client.
 Here we manually supply the name and json parameters for the call.
 
+The `call` procedure takes care of the basic format of the JSON to send to the server.
+However you still need to provide `params` as a `JsonNode`, which must exactly match the parameters defined in the equivalent `rpc` definition.
+
 ```nim
-import rpcclient, asyncdispatch, json
+import rpcclient, rpcserver, asyncdispatch, json
 
-proc main =
+var
-  var client = newRpcClient()
+  server = newRpcSocketServer("localhost", Port(8545))
-  await client.connect("localhost", Port(8545))
+  client = newRpcSocketClient()
-  let response = waitFor client.call("myRpc", %[])
-  # the call returns a `Response` type which contains the result
-  echo response.result.pretty
 
-waitFor main()
+server.start
+
+server.rpc("hello") do(input: string) -> string:
+  result = "Hello " & input
+
+waitFor client.connect("localhost", Port(8545))
+
+let response = waitFor client.call("hello", %[%"Daisy"])
+
+# the call returns a `Response` type which contains the result
+echo response.result
 ```
 
-To make things more readable and allow better checking client side, Json-Rpc supports generating wrappers for client RPCs using `createRpcSigs`.
+## `createRpcSigs`
+
+To make things more readable and allow better static checking client side, Json-Rpc supports generating wrappers for client RPCs using `createRpcSigs`.
+
+This macro takes a type name and the path of a file containing forward declarations of procedures that you wish to convert to client RPCs. The transformation generates procedures that match the forward declarations provided, plus a `client` parameter in the specified type.
 
-This macro takes the path of a file containing forward declarations of procedures that you wish to convert to client RPCs.
 Because the signatures are parsed at compile time, the file will be error checked and you can use import to share common types between your client and server.
 
+### Parameters
+
+  `clientType`: This is the type you want to pass to your generated calls. Usually this would be a transport specific descendant from `RpcClient`.
+  `path`: The path to the Nim module that contains the RPC header signatures.
+
+### Example
+
 For example, to support this remote call:
 
 ```nim
@@ -134,6 +278,37 @@ You can use:
 let bmiIndex = await client.bmi(120.5, 12.0)
 ```
 
+This allows you to leverage Nim's static type checking whilst also aiding readability and providing a unified location to declare client side RPC definitions.
+
+## Working with client transports
+
+Transport clients should provide a type that is inherited from `RpcClient` where they can store any transport related information.
+
+Additionally, the following two procedures are useful:
+
+* `Call`
+  `self`: a descendant of `RpcClient`
+  `name: string`: the method to be called
+  `params: JsonNode`: The parameters to the RPC call
+  Returning
+    `Future[Response]`: A wrapper for the result `JsonNode` and a flag to indicate if this contains an error.
+
+Note: Although `call` isn't necessary for a client to function, it allows RPC signatures to be used by the `createRpcSigs`.
+
+* `Connect`
+  `client`: a descendant of `RpcClient`
+  Returning
+    `FutureBase`: The base future returned when a procedure is annoted with `{.async.}`
+
+### `processMessage`
+
+To simplify and unify processing within the client, the `processMessage` procedure can be used to perform conversion and error checking from the received string originating from the transport to `JsonNode`.
+
+`processMessages`
+  `self`: a client type descended from `RpcClient`
+  `line: string`: a string that contains the JSON to be processed
+
+This procedure then completes the futures set by `call` invocations using the `id` field of the processed `JsonNode` from `line`.
 
 # Contributing
 Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
@@ -142,3 +317,5 @@ Please make sure to update tests as appropriate.
 
 # License
 [MIT](https://choosealicense.com/licenses/mit/)
+or
+Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)