Update documentation

README.md

@@ -3,110 +3,254 @@
 [![Build Status (Travis)](https://img.shields.io/travis/status-im/nim-eth-rpc/master.svg?label=Linux%20/%20macOS "Linux/macOS build status (Travis)")](https://travis-ci.org/status-im/nim-eth-rpc)
 [![Windows build status (Appveyor)](https://img.shields.io/appveyor/ci/jarradh/nim-eth-rpc/master.svg?label=Windows "Windows build status (Appveyor)")](https://ci.appveyor.com/project/jarradh/nim-eth-rpc)
 
-Json-Rpc is designed to provide an easier interface for working with remote procedure calls.
+Json-Rpc is a library designed to provide an easier interface for working with remote procedure calls.
 
 # Installation
 
 `git clone https://github.com/status-im/nim-eth-rpc`
 
 
-## Requirements
+### Requirements
 * Nim 17.3 and up
 
 
-# Usage
+# Introduction
 
-## Server
+Json-Rpc is a library for routing JSON 2.0 format remote procedure calls over different transports.
+It is designed to automatically generate marshalling and parameter checking code based on the RPC parameter types.
 
-Remote procedure calls are created using the `rpc` macro.
+## Routing
-This macro allows you to provide a list of native Nim type parameters and a return type, and will automatically handle all the marshalling to and from json for you, so you can concentrate on using native Nim types for your call.
 
-Here's a full example of a server with a single RPC.
+Remote procedure calls are created using the `rpc` macro on an instance of `RpcRouter`.
+
+`rpc` allows you to provide a list of native Nim type parameters and a return type, generates marshalling to and from json for you, so you can concentrate on using native Nim types for your call.
+
+Routing is then performed by the `route` procedure.
+
+When an error occurs, the `error` is populated, otherwise `result` will be populated.
+
+### Parameters
+
+`route`
+  `path`: The string to match for the `method`.
+  `body`: The parameters and code to execute for this call.
+
+### Example
+
+Here's a simple example:
 
 ```nim
 import rpcserver
 
-var srv = newRpcServer("")
+var router = newRpcRouter()
 
-# Create an RPC with a string an array parameter, that returns an int
+router.rpc:
-srv.rpc("myProc") do(input: int, data: array[0..3, int]) -> string:
+  result = %"Hello"
-  result = "Hello " & $input & " data: " & $data
-
-asyncCheck srv.serve()
-runForever()
 ```
 
-Parameter types are recursively traversed so you can use any custom types you wish, even nested types. Ref and object types are fully supported.
+As no return type was specified in this example, `result` defaults to the `JsonNode` type.
+A JSON string is returned by passing a string though the `%` operator, which converts simple types to `JsonNode`.
+
+The `body` parameters can be defined by using [do notation](https://nim-lang.org/docs/manual.html#procedures-do-notation).
+This allows full Nim types to be used as RPC parameters.
+
+Here we pass a string to an RPC and return a string.
+
+```nim
+router.rpc("hello") do(input: string) -> string:
+  result = "Hello " & input
+```
+
+Json-Rpc will recursively parse the Nim types in order to produce marshalling code.
+This marshalling code uses the types to check the incoming JSON fields to ensure they exist and are of the correct kind.
+
+The return type then performs the opposite process, converting Nim types to Json for transport.
+
+Here is a more complex parameter example:
 
 ```nim
 type
-  Details = ref object
+  HeaderKind = enum hkOne, hkTwo, hkThree
-    values: seq[byte]
-
-  Payload = object
-    x, y: float
-    count: int
-    details: Details
   
-  ResultData = object
+  Header = ref object
-    data: array[10, byte]
+    kind: HeaderKind
+    size: int64
 
-srv.rpc("getResults") do(payload: Payload) -> ResultData:
+  DataBlob = object
-  # Here we can use Payload as expected, and `result` will be of type ResultData.
+    items: seq[byte]
-  # Parameters and results are automatically converted to and from json
+    headers: array[3, Header]
-  # and the call is intrinsically asynchronous.
+
-  
+  MyObject = object
+    data: DataBlob
+    name: string
+
+router.rpc("updateData") do(myObj: MyObject, newData: DataBlob) -> DataBlob:
+  result = myObj.data  
+  myObj.data = newData
 ```
 
-Behind the scenes, all RPC calls take a single json parameter that must be defined as a `JArray`.
+Behind the scenes, all RPC calls take a single json parameter `param` that must be of kind `JArray`.
 At runtime, the json is checked to ensure that it contains the correct number and type of your parameters to match the `rpc` definition.
-The `rpc` macro takes care of the boiler plate in marshalling to and from json.
 
-Compiling with `-d:nimDumpRpcs` will show the output code for the RPC call.
+Compiling with `-d:nimDumpRpcs` will show the output code for the RPC call. To see the output of the `async` generation, add `-d:nimDumpAsync`.
 
-The following RPC:
+## Format
 
-```nim
+The router expects either a string or `JsonNode` with the following structure:
-srv.rpc("myProc") do(input: string, data: array[0..3, int]):
-  result = %("Hello " & input & " data: " & $data)
-```
-Will get transformed into something like this:
 
-```nim
+```json
-proc myProc*(params: JsonNode): Future[JsonNode] {.async.} =
+{
-  params.kind.expect(JArray, "params")
+  "id": JInt,
-  if params.len != 2:
+  "jsonrpc": "2.0",
-    raise newException(ValueError, "Expected 2 Json parameter(s) but got " &
+  "method": JString,
-        $params.len)
+  "params": JArray
-  var input: string
+}
-  input = unpackArg(params.elems[0], "input", type(string))
-  var data: array[0 .. 3, int]
-  data = unpackArg(params.elems[1], "data", type(array[0 .. 3, int]))
-  result = %("Hello " & input & " data: " & $data)
 ```
 
-## Client
+Return values use the following node structure:
+
+```json
+{
+  "id": JInt,
+  "jsonrpc": "2.0",
+  "result": JsonNode,
+  "error": JsonNode
+}
+```
+
+## Performing a route
+
+To call the router, use the `route` procedure.
+
+There are three variants of `route`.
+
+Note that once invoked all RPC calls are error trapped and any exceptions raised are passed back with the error message encoded as a `JsonNode`.
+
+  `route`
+    `router: RpcRouter`: The router object that contains the RPCs.
+    `data: string`: A string ready to be processed into a `JsonNode`.
+    Returns
+    `Future[string]`: This will be the stringified JSON response, which can be the JSON RPC result or a JSON wrapped error.
+  
+  This `route` variant will handle all the conversion of `string` to `JsonNode` and check the format and type of the input data.
+
+  `route`
+    `router: RpcRouter`: The router object that contains the RPCs.
+    `node: JsonNode`: A pre-processed JsonNode that matches the expected format as defined above.
+    Returns
+    `Future[JsonNode]`: The JSON RPC result or a JSON wrapped error.
+  
+  This variant allows simplified processing if you already have a `JsonNode`. However if the required fields are not present within `node`, exceptions will be raised.
+
+  `tryRoute`
+    `router: RpcRouter`: The router object that contains the RPCs.
+    `node: JsonNode`: A pre-processed JsonNode that matches the expected format as defined above.
+    `fut: var Future[JsonNode]`: The JSON RPC result or a JSON wrapped error.
+    Returns
+    `bool`: Returns `true` if the `method` field provided in `node` matches an available route. Returns `false` when the `method` cannot be found, or if `method` or `params` field cannot be found within `node`.
+  
+  This `route` variant allows you to invoke a call if possible, without raising an exception.
+
+To see the result of a call, we need to provide Json in the expected format.
+Here's an example of how that looks by manually creating the JSON. Later we will see the helper utilities that make this easier.
+
+```nim
+let call = %*{
+  "id": %1,
+  "jsonrpc": %2.0,
+  "method": %"hello",
+  "params": %["Terry"]
+  }
+# route the call we defined earlier
+let localResult = waitFor router.route(call)
+
+echo localResult
+# We should see something like this
+#   {"jsonrpc":"2.0","id":1,"result":"Hello Terry","error":null}
+```
+
+# Server
+
+In order to make routing useful, RPCs must be invoked and transmitted over a transport.
+
+The `RpcServer` type is given as a simple inheritable wrapper/container that simplifies designing your own transport layers using the `router` field.
+
+## Server Transports
+
+Currently there are plans for the following transports to be implemented:
+
+* [x] Sockets
+* [ ] HTTP
+* [ ] IPC
+* [ ] Websockets
+
+Transport specific server need only call the `route` procedure using a string fetched from the transport in order to invoke the requested RPC.
+
+## Server example
+
+This example uses the socket transport defined in `socket.nim`.
+Once executed, the "hello" RPC will be available to a socket based client.
+
+```nim
+import rpcserver
+
+# Create a socket server for transport
+var srv = newRpcSocketServer("localhost", Port(8585))
+
+# srv.rpc is a shortcut for srv.router.rpc 
+srv.rpc("hello") do(input: string) -> string:
+  result = "Hello " & input
+
+srv.start()
+runForever()
+```
+
+# Client
+
+Json-Rpc also comes with a client implementation, built to provide a framework for transports to work with.
+
+To simplify demonstration, we will use the socket transport defined in `socketclient.nim`.
 
 Below is the most basic way to use a remote call on the client.
-Here we manually supply the name and json parameters for the call. 
+Here we manually supply the name and json parameters for the call.
+
+The `call` procedure takes care of the basic format of the JSON to send to the server.
+However you still need to provide `params` as a `JsonNode`, which must exactly match the parameters defined in the equivalent `rpc` definition.
 
 ```nim
-import rpcclient, asyncdispatch, json
+import rpcclient, rpcserver, asyncdispatch, json
 
-proc main =
+var
-  var client = newRpcClient()
+  server = newRpcSocketServer("localhost", Port(8545))
-  await client.connect("localhost", Port(8545))
+  client = newRpcSocketClient()
-  let response = waitFor client.call("myRpc", %[])
-  # the call returns a `Response` type which contains the result
-  echo response.result.pretty
 
-waitFor main()
+server.start
+
+server.rpc("hello") do(input: string) -> string:
+  result = "Hello " & input
+
+waitFor client.connect("localhost", Port(8545))
+
+let response = waitFor client.call("hello", %[%"Daisy"])
+
+# the call returns a `Response` type which contains the result
+echo response.result
 ```
 
-To make things more readable and allow better checking client side, Json-Rpc supports generating wrappers for client RPCs using `createRpcSigs`.
+## `createRpcSigs`
 
-This macro takes the path of a file containing forward declarations of procedures that you wish to convert to client RPCs.
+To make things more readable and allow better static checking client side, Json-Rpc supports generating wrappers for client RPCs using `createRpcSigs`.
-Because the signatures are parsed at compile time, the file will be error checked and you can use import to share common types between your client and server. 
+
+This macro takes a type name and the path of a file containing forward declarations of procedures that you wish to convert to client RPCs. The transformation generates procedures that match the forward declarations provided, plus a `client` parameter in the specified type.
+
+Because the signatures are parsed at compile time, the file will be error checked and you can use import to share common types between your client and server.
+
+### Parameters
+
+  `clientType`: This is the type you want to pass to your generated calls. Usually this would be a transport specific descendant from `RpcClient`.
+  `path`: The path to the Nim module that contains the RPC header signatures.
+
+### Example
 
 For example, to support this remote call:
 
@@ -134,6 +278,37 @@ You can use:
 let bmiIndex = await client.bmi(120.5, 12.0)
 ```
 
+This allows you to leverage Nim's static type checking whilst also aiding readability and providing a unified location to declare client side RPC definitions.
+
+## Working with client transports
+
+Transport clients should provide a type that is inherited from `RpcClient` where they can store any transport related information.
+
+Additionally, the following two procedures are useful:
+
+* `Call`
+  `self`: a descendant of `RpcClient`
+  `name: string`: the method to be called
+  `params: JsonNode`: The parameters to the RPC call
+  Returning
+    `Future[Response]`: A wrapper for the result `JsonNode` and a flag to indicate if this contains an error.
+
+Note: Although `call` isn't necessary for a client to function, it allows RPC signatures to be used by the `createRpcSigs`.
+
+* `Connect`
+  `client`: a descendant of `RpcClient`
+  Returning
+    `FutureBase`: The base future returned when a procedure is annoted with `{.async.}`
+
+### `processMessage`
+
+To simplify and unify processing within the client, the `processMessage` procedure can be used to perform conversion and error checking from the received string originating from the transport to `JsonNode`.
+
+`processMessages`
+  `self`: a client type descended from `RpcClient`
+  `line: string`: a string that contains the JSON to be processed
+
+This procedure then completes the futures set by `call` invocations using the `id` field of the processed `JsonNode` from `line`.
 
 # Contributing
 Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
@@ -141,4 +316,6 @@ Pull requests are welcome. For major changes, please open an issue first to disc
 Please make sure to update tests as appropriate.
 
 # License
-[MIT](https://choosealicense.com/licenses/mit/)
+[MIT](https://choosealicense.com/licenses/mit/)
+or
+Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)