Upgrade rpc router internals (#178)

* Upgrade rpc router internals

* use new chronos asyncraises

* Fix style mismatch

* Fix nim v2 compilation error

* Addresing review

* Remove unnecessary custom serializer and let the library do the work

* fix error message

* Update readme.md
This commit is contained in:
andri lim 2024-01-03 20:06:53 +07:00 committed by GitHub
parent c3769f9130
commit e0b077fea4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
37 changed files with 1904 additions and 793 deletions

103
README.md
View File

@ -49,7 +49,7 @@ router.rpc("hello") do() -> string:
``` ```
As no return type was specified in this example, `result` defaults to the `JsonNode` type. As no return type was specified in this example, `result` defaults to the `JsonNode` type.
A JSON string is returned by passing a string though the `%` operator, which converts simple types to `JsonNode`. A JSON string is returned by passing a string though the `JrpcConv` converter powered by [nim-json-serialization](https://github.com/status-im/nim-json-serialization).
The `body` parameters can be defined by using [do notation](https://nim-lang.org/docs/manual.html#procedures-do-notation). The `body` parameters can be defined by using [do notation](https://nim-lang.org/docs/manual.html#procedures-do-notation).
This allows full Nim types to be used as RPC parameters. This allows full Nim types to be used as RPC parameters.
@ -89,7 +89,7 @@ router.rpc("updateData") do(myObj: MyObject, newData: DataBlob) -> DataBlob:
myObj.data = newData myObj.data = newData
``` ```
Behind the scenes, all RPC calls take a single json parameter `param` that must be of kind `JArray`. Behind the scenes, all RPC calls take parameters through `RequestParamsRx` structure.
At runtime, the json is checked to ensure that it contains the correct number and type of your parameters to match the `rpc` definition. At runtime, the json is checked to ensure that it contains the correct number and type of your parameters to match the `rpc` definition.
Compiling with `-d:nimDumpRpcs` will show the output code for the RPC call. To see the output of the `async` generation, add `-d:nimDumpAsync`. Compiling with `-d:nimDumpRpcs` will show the output code for the RPC call. To see the output of the `async` generation, add `-d:nimDumpAsync`.
@ -129,83 +129,71 @@ type
Note that `array` parameters are explicitly checked for length, and will return an error node if the length differs from their declaration size. Note that `array` parameters are explicitly checked for length, and will return an error node if the length differs from their declaration size.
If you wish to support custom types in a particular way, you can provide matching `fromJson` and `%` procedures. If you wish to support custom types in a particular way, you can provide matching `readValue` and `writeValue` procedures.
The custom serializer you write must be using `JrpcConv` flavor.
### `fromJson` ### `readValue`
This takes a Json type and returns the Nim type. This takes a Json type and returns the Nim type.
#### Parameters #### Parameters
`n: JsonNode`: The current node being processed `r: var JsonReader[JrpcConv]`: The current JsonReader with JrpcConv flavor.
`argName: string`: The name of the field in `n` `val: var MyInt`: Deserialized value.
`result`: The type of this must be `var X` where `X` is the Nim type you wish to handle
#### Example #### Example
```nim ```nim
proc fromJson[T](n: JsonNode, argName: string, result: var seq[T]) = proc readValue*(r: var JsonReader[JrpcConv], val: var MyInt)
n.kind.expect(JArray, argName) {.gcsafe, raises: [IOError, JsonReaderError].} =
result = newSeq[T](n.len) let intVal = r.parseInt(int)
for i in 0 ..< n.len: val = MyInt(intVal)
fromJson(n[i], argName, result[i])
``` ```
### `%` ### `writeValue`
This is the standard way to provide translations from a Nim type to a `JsonNode`. This is the standard way to provide translations from a Nim type to Json.
#### Parameters #### Parameters
`n`: The type you wish to convert `w: var JsonWriter[JrpcConv]`: The current JsonWriter with JrpcConv flavor.
#### Returns `val: MyInt`: The value you want to convert into Json.
`JsonNode`: The newly encoded `JsonNode` type from the parameter type.
### `expect`
This is a simple procedure to state your expected type.
If the actual type doesn't match the expected type, an exception is thrown mentioning which field caused the failure.
#### Parameters
`actual: JsonNodeKind`: The actual type of the `JsonNode`.
`expected: JsonNodeKind`: The desired type.
`argName: string`: The current field name.
#### Example #### Example
```nim ```nim
myNode.kind.expect(JArray, argName) proc writeValue*(w: var JsonWriter[JrpcConv], val: MyInt)
{.gcsafe, raises: [IOError].} =
w.writeValue val.int
``` ```
## JSON Format ## JSON Format
The router expects either a string or `JsonNode` with the following structure: The router expects either a Json document with the following structure:
```json ```json
{ {
"id": JInt, "id": Int or String,
"jsonrpc": "2.0", "jsonrpc": "2.0",
"method": JString, "method": String,
"params": JArray "params": Array or Object
} }
``` ```
If params is an Array, it is a positional parameters. If it is an Object then the rpc method will be called using named parameters.
Return values use the following node structure: Return values use the following node structure:
```json ```json
{ {
"id": JInt, "id": Int Or String,
"jsonrpc": "2.0", "jsonrpc": "2.0",
"result": JsonNode, "result": Json document,
"error": JsonNode "error": Json document
} }
``` ```
@ -215,35 +203,35 @@ To call and RPC through the router, use the `route` procedure.
There are three variants of `route`. There are three variants of `route`.
Note that once invoked all RPC calls are error trapped and any exceptions raised are passed back with the error message encoded as a `JsonNode`. Note that once invoked all RPC calls are error trapped and any exceptions raised are passed back with the error message encoded as a `Json document`.
### `route` by string ### `route` by string
This `route` variant will handle all the conversion of `string` to `JsonNode` and check the format and type of the input data. This `route` variant will handle all the conversion of `string` to `Json document` and check the format and type of the input data.
#### Parameters #### Parameters
`router: RpcRouter`: The router object that contains the RPCs. `router: RpcRouter`: The router object that contains the RPCs.
`data: string`: A string ready to be processed into a `JsonNode`. `data: string`: A string ready to be processed into a `Json document`.
#### Returns #### Returns
`Future[string]`: This will be the stringified JSON response, which can be the JSON RPC result or a JSON wrapped error. `Future[string]`: This will be the stringified JSON response, which can be the JSON RPC result or a JSON wrapped error.
### `route` by `JsonNode` ### `route` by `Json document`
This variant allows simplified processing if you already have a `JsonNode`. However if the required fields are not present within `node`, exceptions will be raised. This variant allows simplified processing if you already have a `Json document`. However if the required fields are not present within `data`, exceptions will be raised.
#### Parameters #### Parameters
`router: RpcRouter`: The router object that contains the RPCs. `router: RpcRouter`: The router object that contains the RPCs.
`node: JsonNode`: A pre-processed `JsonNode` that matches the expected format as defined above. `req: RequestTx`: A pre-processed `Json document` that matches the expected format as defined above.
#### Returns #### Returns
`Future[JsonNode]`: The JSON RPC result or a JSON wrapped error. `Future[ResponseTx]`: The JSON RPC result or a JSON wrapped error.
### `tryRoute` ### `tryRoute`
@ -253,13 +241,13 @@ This `route` variant allows you to invoke a call if possible, without raising an
`router: RpcRouter`: The router object that contains the RPCs. `router: RpcRouter`: The router object that contains the RPCs.
`node: JsonNode`: A pre-processed `JsonNode` that matches the expected format as defined above. `data: StringOfJson`: A raw `Json document` that matches the expected format as defined above.
`fut: var Future[JsonNode]`: The JSON RPC result or a JSON wrapped error. `fut: var Future[StringOfJson]`: The stringified JSON RPC result or a JSON wrapped error.
#### Returns #### Returns
`bool`: `true` if the `method` field provided in `node` matches an available route. Returns `false` when the `method` cannot be found, or if `method` or `params` field cannot be found within `node`. `Result[void, string]` `isOk` if the `method` field provided in `data` matches an available route. Returns `isErr` when the `method` cannot be found, or if `method` or `params` field cannot be found within `data`.
To see the result of a call, we need to provide Json in the expected format. To see the result of a call, we need to provide Json in the expected format.
@ -326,7 +314,7 @@ Below is the most basic way to use a remote call on the client.
Here we manually supply the name and json parameters for the call. Here we manually supply the name and json parameters for the call.
The `call` procedure takes care of the basic format of the JSON to send to the server. The `call` procedure takes care of the basic format of the JSON to send to the server.
However you still need to provide `params` as a `JsonNode`, which must exactly match the parameters defined in the equivalent `rpc` definition. However you still need to provide `params` as a `JsonNode` or `RequestParamsTx`, which must exactly match the parameters defined in the equivalent `rpc` definition.
```nim ```nim
import json_rpc/[rpcclient, rpcserver], chronos, json import json_rpc/[rpcclient, rpcserver], chronos, json
@ -362,6 +350,11 @@ Because the signatures are parsed at compile time, the file will be error checke
`path`: The path to the Nim module that contains the RPC header signatures. `path`: The path to the Nim module that contains the RPC header signatures.
#### Variants of createRpcSigs
- `createRpcSigsFromString`, generate rpc wrapper from string instead load it from file.
- `createSingleRpcSig`, generate rpc wrapper from single Nim proc signature, with alias. e.g. calling same rpc method using different return type.
- `createRpcSigsFromNim`, generate rpc wrapper from a list Nim proc signature, without loading any file.
#### Example #### Example
For example, to support this remote call: For example, to support this remote call:
@ -404,7 +397,7 @@ Additionally, the following two procedures are useful:
`name: string`: the method to be called `name: string`: the method to be called
`params: JsonNode`: The parameters to the RPC call `params: JsonNode`: The parameters to the RPC call
Returning Returning
`Future[Response]`: A wrapper for the result `JsonNode` and a flag to indicate if this contains an error. `Future[StringOfJson]`: A wrapper for the result `Json document` and a flag to indicate if this contains an error.
Note: Although `call` isn't necessary for a client to function, it allows RPC signatures to be used by the `createRpcSigs`. Note: Although `call` isn't necessary for a client to function, it allows RPC signatures to be used by the `createRpcSigs`.
@ -416,9 +409,9 @@ Note: Although `call` isn't necessary for a client to function, it allows RPC si
### `processMessage` ### `processMessage`
To simplify and unify processing within the client, the `processMessage` procedure can be used to perform conversion and error checking from the received string originating from the transport to the `JsonNode` representation that is passed to the RPC. To simplify and unify processing within the client, the `processMessage` procedure can be used to perform conversion and error checking from the received string originating from the transport to the `Json document` representation that is passed to the RPC.
After a RPC returns, this procedure then completes the futures set by `call` invocations using the `id` field of the processed `JsonNode` from `line`. After a RPC returns, this procedure then completes the futures set by `call` invocations using the `id` field of the processed `Json document` from `line`.
#### Parameters #### Parameters

View File

@ -21,8 +21,8 @@ requires "nim >= 1.6.0",
"stew", "stew",
"nimcrypto", "nimcrypto",
"stint", "stint",
"chronos", "chronos#head",
"httputils", "httputils#head",
"chronicles", "chronicles",
"websock", "websock",
"json_serialization", "json_serialization",

View File

@ -1,5 +1,5 @@
# json-rpc # json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH # Copyright (c) 2019-2024 Status Research & Development GmbH
# Licensed under either of # Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE)) # * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT)) # * MIT license ([LICENSE-MIT](LICENSE-MIT))
@ -8,195 +8,129 @@
# those terms. # those terms.
import import
std/[tables, macros], std/[json, tables, macros],
chronos, chronos,
./jsonmarshal results,
./private/jrpc_conv,
./private/jrpc_sys,
./private/client_handler_wrapper,
./private/shared_wrapper,
./private/errors
from strutils import toLowerAscii, replace from strutils import replace
export export
chronos, jsonmarshal, tables chronos,
tables,
jrpc_conv,
RequestParamsTx,
results
type type
ClientId* = int64
MethodHandler* = proc (j: JsonNode) {.gcsafe, raises: [Defect, CatchableError].}
RpcClient* = ref object of RootRef RpcClient* = ref object of RootRef
awaiting*: Table[ClientId, Future[Response]] awaiting*: Table[RequestId, Future[StringOfJson]]
lastId: ClientId lastId: int
methodHandlers: Table[string, MethodHandler] onDisconnect*: proc() {.gcsafe, raises: [].}
onDisconnect*: proc() {.gcsafe, raises: [Defect].}
Response* = JsonNode GetJsonRpcRequestHeaders* = proc(): seq[(string, string)] {.gcsafe, raises: [].}
GetJsonRpcRequestHeaders* = proc(): seq[(string, string)] {.gcsafe, raises: [Defect].} {.push gcsafe, raises: [].}
proc getNextId*(client: RpcClient): ClientId = # ------------------------------------------------------------------------------
# Public helpers
# ------------------------------------------------------------------------------
func requestTxEncode*(name: string, params: RequestParamsTx, id: RequestId): string =
let req = requestTx(name, params, id)
JrpcSys.encode(req)
# ------------------------------------------------------------------------------
# Public functions
# ------------------------------------------------------------------------------
proc getNextId*(client: RpcClient): RequestId =
client.lastId += 1 client.lastId += 1
client.lastId RequestId(kind: riNumber, num: client.lastId)
proc rpcCallNode*(path: string, params: JsonNode, id: ClientId): JsonNode =
%{"jsonrpc": %"2.0", "method": %path, "params": params, "id": %id}
method call*(client: RpcClient, name: string, method call*(client: RpcClient, name: string,
params: JsonNode): Future[Response] {.base, async.} = params: RequestParamsTx): Future[StringOfJson]
discard {.base, gcsafe, async.} =
doAssert(false, "`RpcClient.call` not implemented")
method close*(client: RpcClient): Future[void] {.base, async.} = method call*(client: RpcClient, name: string,
discard params: JsonNode): Future[StringOfJson]
{.base, gcsafe, async.} =
template `or`(a: JsonNode, b: typed): JsonNode = await client.call(name, params.paramsTx)
if a.isNil: b else: a
proc processMessage*(self: RpcClient, line: string) = method close*(client: RpcClient): Future[void] {.base, gcsafe, async.} =
doAssert(false, "`RpcClient.close` not implemented")
proc processMessage*(client: RpcClient, line: string): Result[void, string] =
# Note: this doesn't use any transport code so doesn't need to be # Note: this doesn't use any transport code so doesn't need to be
# differentiated. # differentiated.
let node = try: parseJson(line) try:
except CatchableError as exc: raise exc let response = JrpcSys.decode(line, ResponseRx)
# TODO https://github.com/status-im/nimbus-eth2/issues/2430
except Exception as exc: raise (ref ValueError)(msg: exc.msg, parent: exc)
if "id" in node: if response.jsonrpc.isNone:
let id = node{"id"} or newJNull() return err("missing or invalid `jsonrpc`")
var requestFut: Future[Response] if response.id.isNone:
if not self.awaiting.pop(id.getInt(-1), requestFut): return err("missing or invalid response id")
raise newException(ValueError, "Cannot find message id \"" & $id & "\"")
let version = node{"jsonrpc"}.getStr() var requestFut: Future[StringOfJson]
if version != "2.0": let id = response.id.get
requestFut.fail(newException(ValueError, if not client.awaiting.pop(id, requestFut):
"Unsupported version of JSON, expected 2.0, received \"" & version & "\"")) return err("Cannot find message id \"" & $id & "\"")
else:
let result = node{"result"}
if result.isNil:
let error = node{"error"} or newJNull()
requestFut.fail(newException(ValueError, $error))
else:
requestFut.complete(result)
elif "method" in node:
# This could be subscription notification
let name = node["method"].getStr()
let handler = self.methodHandlers.getOrDefault(name)
if not handler.isNil:
handler(node{"params"} or newJArray())
else:
raise newException(ValueError, "Invalid jsonrpc message: " & $node)
if response.error.isSome:
let error = JrpcSys.encode(response.error.get)
requestFut.fail(newException(JsonRpcError, error))
return ok()
if response.result.isNone:
return err("missing or invalid response result")
requestFut.complete(response.result.get)
return ok()
except CatchableError as exc:
return err(exc.msg)
# ------------------------------------------------------------------------------
# Signature processing # Signature processing
# ------------------------------------------------------------------------------
proc createRpcProc(procName, parameters, callBody: NimNode): NimNode =
# parameters come as a tree
var paramList = newSeq[NimNode]()
for p in parameters: paramList.add(p)
let body = quote do:
{.gcsafe.}:
`callBody`
# build proc
result = newProc(procName, paramList, body)
# make proc async
result.addPragma ident"async"
# export this proc
result[0] = nnkPostfix.newTree(ident"*", newIdentNode($procName))
proc toJsonArray(parameters: NimNode): NimNode =
# outputs an array of jsonified parameters
# ie; %[%a, %b, %c]
parameters.expectKind nnkFormalParams
var items = newNimNode(nnkBracket)
for i in 2 ..< parameters.len:
let curParam = parameters[i][0]
if curParam.kind != nnkEmpty:
items.add(nnkPrefix.newTree(ident"%", curParam))
result = nnkPrefix.newTree(bindSym("%", brForceOpen), items)
proc createRpcFromSig*(clientType, rpcDecl: NimNode): NimNode =
# Each input parameter in the rpc signature is converted
# to json with `%`.
# Return types are then converted back to native Nim types.
let iJsonNode = newIdentNode("JsonNode")
var parameters = rpcDecl.findChild(it.kind == nnkFormalParams).copy
# ensure we have at least space for a return parameter
if parameters.isNil or parameters.kind == nnkEmpty or parameters.len == 0:
parameters = nnkFormalParams.newTree(iJsonNode)
let
procName = rpcDecl.name
pathStr = $procName
returnType =
# if no return type specified, defaults to JsonNode
if parameters[0].kind == nnkEmpty: iJsonNode
else: parameters[0]
customReturnType = returnType != iJsonNode
# insert rpc client as first parameter
parameters.insert(1, nnkIdentDefs.newTree(ident"client", ident($clientType),
newEmptyNode()))
let
# variable used to send json to the server
jsonParamIdent = genSym(nskVar, "jsonParam")
# json array of marshalled parameters
jsonParamArray = parameters.toJsonArray()
var
# populate json params - even rpcs with no parameters have an empty json
# array node sent
callBody = newStmtList().add(quote do:
var `jsonParamIdent` = `jsonParamArray`
)
# convert return type to Future
parameters[0] = nnkBracketExpr.newTree(ident"Future", returnType)
let
# temporary variable to hold `Response` from rpc call
rpcResult = genSym(nskLet, "res")
clientIdent = newIdentNode("client")
# proc return variable
procRes = ident"result"
# perform rpc call
callBody.add(quote do:
# `rpcResult` is of type `Response`
let `rpcResult` = await `clientIdent`.call(`pathStr`, `jsonParamIdent`)
)
if customReturnType:
# marshal json to native Nim type
callBody.add(jsonToNim(procRes, returnType, rpcResult, "result"))
else:
# native json expected so no work
callBody.add quote do:
`procRes` = if `rpcResult`.isNil:
newJNull()
else:
`rpcResult`
# create rpc proc
result = createRpcProc(procName, parameters, callBody)
when defined(nimDumpRpcs):
echo pathStr, ":\n", result.repr
proc processRpcSigs(clientType, parsedCode: NimNode): NimNode =
result = newStmtList()
for line in parsedCode:
if line.kind == nnkProcDef:
var procDef = createRpcFromSig(clientType, line)
result.add(procDef)
proc setMethodHandler*(cl: RpcClient, name: string, callback: MethodHandler) =
cl.methodHandlers[name] = callback
proc delMethodHandler*(cl: RpcClient, name: string) =
cl.methodHandlers.del(name)
macro createRpcSigs*(clientType: untyped, filePath: static[string]): untyped = macro createRpcSigs*(clientType: untyped, filePath: static[string]): untyped =
## Takes a file of forward declarations in Nim and builds them into RPC ## Takes a file of forward declarations in Nim and builds them into RPC
## calls, based on their parameters. ## calls, based on their parameters.
## Inputs are marshalled to json, and results are put into the signature's ## Inputs are marshalled to json, and results are put into the signature's
## Nim type. ## Nim type.
result = processRpcSigs(clientType, staticRead($filePath.replace('\\', '/')).parseStmt()) cresteSignaturesFromString(clientType, staticRead($filePath.replace('\\', '/')))
macro createRpcSigsFromString*(clientType: untyped, sigString: static[string]): untyped =
## Takes a string of forward declarations in Nim and builds them into RPC
## calls, based on their parameters.
## Inputs are marshalled to json, and results are put into the signature's
## Nim type.
cresteSignaturesFromString(clientType, sigString)
macro createSingleRpcSig*(clientType: untyped, alias: static[string], procDecl: typed): untyped =
## Takes a single forward declarations in Nim and builds them into RPC
## calls, based on their parameters.
## Inputs are marshalled to json, and results are put into the signature's
## Nim type.
doAssert procDecl.len == 1, "Only accept single proc definition"
let procDecl = procDecl[0]
procDecl.expectKind nnkProcDef
result = createRpcFromSig(clientType, procDecl, ident(alias))
macro createRpcSigsFromNim*(clientType: untyped, procList: typed): untyped =
## Takes a list of forward declarations in Nim and builds them into RPC
## calls, based on their parameters.
## Inputs are marshalled to json, and results are put into the signature's
## Nim type.
processRpcSigs(clientType, procList)
{.pop.}

View File

@ -9,16 +9,17 @@
import import
std/[tables, uri], std/[tables, uri],
stew/[byteutils, results], stew/byteutils,
results,
chronos/apps/http/httpclient as chronosHttpClient, chronos/apps/http/httpclient as chronosHttpClient,
chronicles, httputils, json_serialization/std/net, chronicles, httputils, json_serialization/std/net,
".."/[client, errors] ../client,
../private/errors,
../private/jrpc_sys
export export
client, HttpClientFlag, HttpClientFlags client, HttpClientFlag, HttpClientFlags
{.push raises: [Defect].}
logScope: logScope:
topics = "JSONRPC-HTTP-CLIENT" topics = "JSONRPC-HTTP-CLIENT"
@ -35,6 +36,8 @@ type
const const
MaxHttpRequestSize = 128 * 1024 * 1024 # maximum size of HTTP body in octets MaxHttpRequestSize = 128 * 1024 * 1024 # maximum size of HTTP body in octets
{.push gcsafe, raises: [].}
proc new( proc new(
T: type RpcHttpClient, maxBodySize = MaxHttpRequestSize, secure = false, T: type RpcHttpClient, maxBodySize = MaxHttpRequestSize, secure = false,
getHeaders: GetJsonRpcRequestHeaders = nil, flags: HttpClientFlags = {}): T = getHeaders: GetJsonRpcRequestHeaders = nil, flags: HttpClientFlags = {}): T =
@ -51,7 +54,7 @@ proc newRpcHttpClient*(
RpcHttpClient.new(maxBodySize, secure, getHeaders, flags) RpcHttpClient.new(maxBodySize, secure, getHeaders, flags)
method call*(client: RpcHttpClient, name: string, method call*(client: RpcHttpClient, name: string,
params: JsonNode): Future[Response] params: RequestParamsTx): Future[StringOfJson]
{.async, gcsafe.} = {.async, gcsafe.} =
doAssert client.httpSession != nil doAssert client.httpSession != nil
if client.httpAddress.isErr: if client.httpAddress.isErr:
@ -66,7 +69,7 @@ method call*(client: RpcHttpClient, name: string,
let let
id = client.getNextId() id = client.getNextId()
reqBody = $rpcCallNode(name, params, id) reqBody = requestTxEncode(name, params, id)
var req: HttpClientRequestRef var req: HttpClientRequestRef
var res: HttpClientResponseRef var res: HttpClientResponseRef
@ -128,19 +131,18 @@ method call*(client: RpcHttpClient, name: string,
# completed by processMessage - the flow is quite weird here to accomodate # completed by processMessage - the flow is quite weird here to accomodate
# socket and ws clients, but could use a more thorough refactoring # socket and ws clients, but could use a more thorough refactoring
var newFut = newFuture[Response]() var newFut = newFuture[StringOfJson]()
# add to awaiting responses # add to awaiting responses
client.awaiting[id] = newFut client.awaiting[id] = newFut
try: # Might error for all kinds of reasons
# Might raise for all kinds of reasons let msgRes = client.processMessage(resText)
client.processMessage(resText) if msgRes.isErr:
except CatchableError as e:
# Need to clean up in case the answer was invalid # Need to clean up in case the answer was invalid
debug "Failed to process POST Response for JSON-RPC", e = e.msg debug "Failed to process POST Response for JSON-RPC", msg = msgRes.error
client.awaiting.del(id) client.awaiting.del(id)
closeRefs() closeRefs()
raise e raise newException(JsonRpcError, msgRes.error)
client.awaiting.del(id) client.awaiting.del(id)
@ -175,3 +177,5 @@ proc connect*(client: RpcHttpClient, address: string, port: Port, secure: bool)
method close*(client: RpcHttpClient) {.async.} = method close*(client: RpcHttpClient) {.async.} =
if not client.httpSession.isNil: if not client.httpSession.isNil:
await client.httpSession.closeWait() await client.httpSession.closeWait()
{.pop.}

View File

@ -9,10 +9,12 @@
import import
std/tables, std/tables,
chronicles,
results,
chronos, chronos,
../client ../client,
../private/errors,
{.push raises: [Defect].} ../private/jrpc_sys
export client export client
@ -24,6 +26,8 @@ type
const defaultMaxRequestLength* = 1024 * 128 const defaultMaxRequestLength* = 1024 * 128
{.push gcsafe, raises: [].}
proc new*(T: type RpcSocketClient): T = proc new*(T: type RpcSocketClient): T =
T() T()
@ -32,16 +36,16 @@ proc newRpcSocketClient*: RpcSocketClient =
RpcSocketClient.new() RpcSocketClient.new()
method call*(self: RpcSocketClient, name: string, method call*(self: RpcSocketClient, name: string,
params: JsonNode): Future[Response] {.async, gcsafe.} = params: RequestParamsTx): Future[StringOfJson] {.async, gcsafe.} =
## Remotely calls the specified RPC method. ## Remotely calls the specified RPC method.
let id = self.getNextId() let id = self.getNextId()
var value = $rpcCallNode(name, params, id) & "\r\n" var value = requestTxEncode(name, params, id) & "\r\n"
if self.transport.isNil: if self.transport.isNil:
raise newException(ValueError, raise newException(JsonRpcError,
"Transport is not initialised (missing a call to connect?)") "Transport is not initialised (missing a call to connect?)")
# completed by processMessage. # completed by processMessage.
var newFut = newFuture[Response]() var newFut = newFuture[StringOfJson]()
# add to awaiting responses # add to awaiting responses
self.awaiting[id] = newFut self.awaiting[id] = newFut
@ -60,8 +64,10 @@ proc processData(client: RpcSocketClient) {.async.} =
await client.transport.closeWait() await client.transport.closeWait()
break break
# TODO handle exceptions let res = client.processMessage(value)
client.processMessage(value) if res.isErr:
error "error when processing message", msg=res.error
raise newException(JsonRpcError, res.error)
# async loop reconnection and waiting # async loop reconnection and waiting
client.transport = await connect(client.address) client.transport = await connect(client.address)

View File

@ -11,13 +11,12 @@ import
std/[uri, strutils], std/[uri, strutils],
pkg/websock/[websock, extensions/compression/deflate], pkg/websock/[websock, extensions/compression/deflate],
pkg/[chronos, chronos/apps/http/httptable, chronicles], pkg/[chronos, chronos/apps/http/httptable, chronicles],
stew/byteutils stew/byteutils,
../private/errors
# avoid clash between Json.encode and Base64Pad.encode # avoid clash between Json.encode and Base64Pad.encode
import ../client except encode import ../client except encode
{.push raises: [Defect].}
logScope: logScope:
topics = "JSONRPC-WS-CLIENT" topics = "JSONRPC-WS-CLIENT"
@ -28,6 +27,8 @@ type
loop*: Future[void] loop*: Future[void]
getHeaders*: GetJsonRpcRequestHeaders getHeaders*: GetJsonRpcRequestHeaders
{.push gcsafe, raises: [].}
proc new*( proc new*(
T: type RpcWebSocketClient, getHeaders: GetJsonRpcRequestHeaders = nil): T = T: type RpcWebSocketClient, getHeaders: GetJsonRpcRequestHeaders = nil): T =
T(getHeaders: getHeaders) T(getHeaders: getHeaders)
@ -38,16 +39,16 @@ proc newRpcWebSocketClient*(
RpcWebSocketClient.new(getHeaders) RpcWebSocketClient.new(getHeaders)
method call*(self: RpcWebSocketClient, name: string, method call*(self: RpcWebSocketClient, name: string,
params: JsonNode): Future[Response] {.async, gcsafe.} = params: RequestParamsTx): Future[StringOfJson] {.async, gcsafe.} =
## Remotely calls the specified RPC method. ## Remotely calls the specified RPC method.
let id = self.getNextId() let id = self.getNextId()
var value = $rpcCallNode(name, params, id) & "\r\n" var value = requestTxEncode(name, params, id) & "\r\n"
if self.transport.isNil: if self.transport.isNil:
raise newException(ValueError, raise newException(JsonRpcError,
"Transport is not initialised (missing a call to connect?)") "Transport is not initialised (missing a call to connect?)")
# completed by processMessage. # completed by processMessage.
var newFut = newFuture[Response]() var newFut = newFuture[StringOfJson]()
# add to awaiting responses # add to awaiting responses
self.awaiting[id] = newFut self.awaiting[id] = newFut
@ -66,7 +67,10 @@ proc processData(client: RpcWebSocketClient) {.async.} =
# transmission ends # transmission ends
break break
client.processMessage(string.fromBytes(value)) let res = client.processMessage(string.fromBytes(value))
if res.isErr:
raise newException(JsonRpcError, res.error)
except CatchableError as e: except CatchableError as e:
error = e error = e

View File

@ -1,242 +0,0 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
std/[macros, json, typetraits],
stew/[byteutils, objects],
json_serialization,
json_serialization/lexer,
json_serialization/std/[options, sets, tables]
export json, options, json_serialization
Json.createFlavor JsonRpc
# Avoid templates duplicating the string in the executable.
const errDeserializePrefix = "Error deserializing stream for type '"
template wrapErrors(reader, value, actions: untyped): untyped =
## Convert read errors to `UnexpectedValue` for the purpose of marshalling.
try:
actions
except Exception as err:
reader.raiseUnexpectedValue(errDeserializePrefix & $type(value) & "': " & err.msg)
# Bytes.
proc readValue*(r: var JsonReader[JsonRpc], value: var byte) =
## Implement separate read serialization for `byte` to avoid
## 'can raise Exception' for `readValue(value, uint8)`.
wrapErrors r, value:
case r.lexer.tok
of tkInt:
if r.lexer.absIntVal in 0'u32 .. byte.high:
value = byte(r.lexer.absIntVal)
else:
r.raiseIntOverflow r.lexer.absIntVal, true
of tkNegativeInt:
r.raiseIntOverflow r.lexer.absIntVal, true
else:
r.raiseUnexpectedToken etInt
r.lexer.next()
proc writeValue*(w: var JsonWriter[JsonRpc], value: byte) =
json_serialization.writeValue(w, uint8(value))
# Enums.
proc readValue*(r: var JsonReader[JsonRpc], value: var (enum)) =
wrapErrors r, value:
value = type(value) json_serialization.readValue(r, uint64)
proc writeValue*(w: var JsonWriter[JsonRpc], value: (enum)) =
json_serialization.writeValue(w, uint64(value))
# Other base types.
macro genDistinctSerializers(types: varargs[untyped]): untyped =
## Implements distinct serialization pass-throughs for `types`.
result = newStmtList()
for ty in types:
result.add(quote do:
proc readValue*(r: var JsonReader[JsonRpc], value: var `ty`) =
wrapErrors r, value:
json_serialization.readValue(r, value)
proc writeValue*(w: var JsonWriter[JsonRpc], value: `ty`) {.raises: [IOError].} =
json_serialization.writeValue(w, value)
)
genDistinctSerializers bool, int, float, string, int64, uint64, uint32, ref int64, ref int
# Sequences and arrays.
proc readValue*[T](r: var JsonReader[JsonRpc], value: var seq[T]) =
wrapErrors r, value:
json_serialization.readValue(r, value)
proc writeValue*[T](w: var JsonWriter[JsonRpc], value: seq[T]) =
json_serialization.writeValue(w, value)
proc readValue*[N: static[int]](r: var JsonReader[JsonRpc], value: var array[N, byte]) =
## Read an array while allowing partial data.
wrapErrors r, value:
r.skipToken tkBracketLe
if r.lexer.tok != tkBracketRi:
for i in low(value) .. high(value):
readValue(r, value[i])
if r.lexer.tok == tkBracketRi:
break
else:
r.skipToken tkComma
r.skipToken tkBracketRi
# High level generic unpacking.
proc unpackArg[T](args: JsonNode, argName: string, argType: typedesc[T]): T {.raises: [ValueError].} =
if args.isNil:
raise newException(ValueError, argName & ": unexpected null value")
try:
result = JsonRpc.decode($args, argType)
except CatchableError as err:
raise newException(ValueError,
"Parameter [" & argName & "] of type '" & $argType & "' could not be decoded: " & err.msg)
proc expect*(actual, expected: JsonNodeKind, argName: string) =
if actual != expected:
raise newException(
ValueError, "Parameter [" & argName & "] expected " & $expected & " but got " & $actual)
proc expectArrayLen(node, jsonIdent: NimNode, length: int) =
let
identStr = jsonIdent.repr
expectedStr = "Expected " & $length & " Json parameter(s) but got "
node.add(quote do:
`jsonIdent`.kind.expect(JArray, `identStr`)
if `jsonIdent`.len != `length`:
raise newException(ValueError, `expectedStr` & $`jsonIdent`.len)
)
iterator paramsIter(params: NimNode): tuple[name, ntype: NimNode] =
for i in 1 ..< params.len:
let arg = params[i]
let argType = arg[^2]
for j in 0 ..< arg.len-2:
yield (arg[j], argType)
iterator paramsRevIter(params: NimNode): tuple[name, ntype: NimNode] =
for i in countdown(params.len-1,1):
let arg = params[i]
let argType = arg[^2]
for j in 0 ..< arg.len-2:
yield (arg[j], argType)
proc isOptionalArg(typeNode: NimNode): bool =
typeNode.kind == nnkBracketExpr and
typeNode[0].kind == nnkIdent and
typeNode[0].strVal == "Option"
proc expectOptionalArrayLen(node, parameters, jsonIdent: NimNode, maxLength: int): int =
var minLength = maxLength
for arg, typ in paramsRevIter(parameters):
if not typ.isOptionalArg: break
dec minLength
let
identStr = jsonIdent.repr
expectedStr = "Expected at least " & $minLength & " and maximum " & $maxLength & " Json parameter(s) but got "
node.add(quote do:
`jsonIdent`.kind.expect(JArray, `identStr`)
if `jsonIdent`.len < `minLength`:
raise newException(ValueError, `expectedStr` & $`jsonIdent`.len)
)
minLength
proc containsOptionalArg(params: NimNode): bool =
for n, t in paramsIter(params):
if t.isOptionalArg:
return true
proc jsonToNim*(assignIdent, paramType, jsonIdent: NimNode, paramNameStr: string, optional = false): NimNode =
# verify input and load a Nim type from json data
# note: does not create `assignIdent`, so can be used for `result` variables
result = newStmtList()
# unpack each parameter and provide assignments
let unpackNode = quote do:
`unpackArg`(`jsonIdent`, `paramNameStr`, type(`paramType`))
if optional:
result.add(quote do: `assignIdent` = some(`unpackNode`))
else:
result.add(quote do: `assignIdent` = `unpackNode`)
proc calcActualParamCount(params: NimNode): int =
# this proc is needed to calculate the actual parameter count
# not matter what is the declaration form
# e.g. (a: U, b: V) vs. (a, b: T)
for n, t in paramsIter(params):
inc result
proc jsonToNim*(params, jsonIdent: NimNode): NimNode =
# Add code to verify input and load params into Nim types
result = newStmtList()
if not params.isNil:
var minLength = 0
if params.containsOptionalArg():
# more elaborate parameters array check
minLength = result.expectOptionalArrayLen(params, jsonIdent,
calcActualParamCount(params))
else:
# simple parameters array length check
result.expectArrayLen(jsonIdent, calcActualParamCount(params))
# unpack each parameter and provide assignments
var pos = 0
for paramIdent, paramType in paramsIter(params):
# processing multiple variables of one type
# e.g. (a, b: T), including common (a: U, b: V) form
let
paramName = $paramIdent
jsonElement = quote do:
`jsonIdent`.elems[`pos`]
# declare variable before assignment
result.add(quote do:
var `paramIdent`: `paramType`
)
# e.g. (A: int, B: Option[int], C: string, D: Option[int], E: Option[string])
if paramType.isOptionalArg:
let
innerType = paramType[1]
innerNode = jsonToNim(paramIdent, innerType, jsonElement, paramName, true)
if pos >= minLength:
# allow both empty and null after mandatory args
# D & E fall into this category
result.add(quote do:
if `jsonIdent`.len > `pos` and `jsonElement`.kind != JNull: `innerNode`
)
else:
# allow null param for optional args between/before mandatory args
# B fall into this category
result.add(quote do:
if `jsonElement`.kind != JNull: `innerNode`
)
else:
# mandatory args
# A and C fall into this category
# unpack Nim type and assign from json
result.add jsonToNim(paramIdent, paramType, jsonElement, paramName)
inc pos

View File

@ -0,0 +1,109 @@
# json-rpc
# Copyright (c) 2024 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
macros,
./shared_wrapper
{.push gcsafe, raises: [].}
proc createRpcProc(procName, parameters, callBody: NimNode): NimNode =
# parameters come as a tree
var paramList = newSeq[NimNode]()
for p in parameters: paramList.add(p)
let body = quote do:
{.gcsafe.}:
`callBody`
# build proc
result = newProc(procName, paramList, body)
# make proc async
result.addPragma ident"async"
result.addPragma ident"gcsafe"
# export this proc
result[0] = nnkPostfix.newTree(ident"*", newIdentNode($procName))
proc setupConversion(reqParams, params: NimNode): NimNode =
# populate json params
# even rpcs with no parameters have an empty json array node sent
params.expectKind nnkFormalParams
result = newStmtList()
result.add quote do:
var `reqParams` = RequestParamsTx(kind: rpPositional)
for parName, parType in paramsIter(params):
result.add quote do:
`reqParams`.positional.add encode(JrpcConv, `parName`).StringOfJson
proc createRpcFromSig*(clientType, rpcDecl: NimNode, alias = NimNode(nil)): NimNode =
# Each input parameter in the rpc signature is converted
# to json using JrpcConv.encode.
# Return types are then converted back to native Nim types.
let
params = rpcDecl.findChild(it.kind == nnkFormalParams).ensureReturnType
procName = if alias.isNil: rpcDecl.name else: alias
pathStr = $rpcDecl.name
returnType = params[0]
reqParams = genSym(nskVar, "reqParams")
setup = setupConversion(reqParams, params)
clientIdent = ident"client"
# temporary variable to hold `Response` from rpc call
rpcResult = genSym(nskLet, "res")
# proc return variable
procRes = ident"result"
doDecode = quote do:
`procRes` = decode(JrpcConv, `rpcResult`.string, typeof `returnType`)
maybeWrap =
if returnType.noWrap: quote do:
`procRes` = `rpcResult`
else: doDecode
# insert rpc client as first parameter
params.insert(1, nnkIdentDefs.newTree(
clientIdent,
ident($clientType),
newEmptyNode()
))
# convert return type to Future
params[0] = nnkBracketExpr.newTree(ident"Future", returnType)
# perform rpc call
let callBody = quote do:
# populate request params
`setup`
# `rpcResult` is of type `StringOfJson`
let `rpcResult` = await `clientIdent`.call(`pathStr`, `reqParams`)
`maybeWrap`
# create rpc proc
result = createRpcProc(procName, params, callBody)
when defined(nimDumpRpcs):
echo pathStr, ":\n", result.repr
proc processRpcSigs*(clientType, parsedCode: NimNode): NimNode =
result = newStmtList()
for line in parsedCode:
if line.kind == nnkProcDef:
var procDef = createRpcFromSig(clientType, line)
result.add(procDef)
proc cresteSignaturesFromString*(clientType: NimNode, sigStrings: string): NimNode =
try:
result = processRpcSigs(clientType, sigStrings.parseStmt())
except ValueError as exc:
doAssert(false, exc.msg)
{.pop.}

View File

@ -31,3 +31,10 @@ type
## This could be raised by request handlers when the server ## This could be raised by request handlers when the server
## needs to respond with a custom error code. ## needs to respond with a custom error code.
code*: int code*: int
RequestDecodeError* = object of JsonRpcError
## raised when fail to decode RequestRx
ParamsEncodeError* = object of JsonRpcError
## raised when fail to encode RequestParamsTx

View File

@ -0,0 +1,23 @@
# json-rpc
# Copyright (c) 2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
json_serialization
export
json_serialization
type
StringOfJson* = JsonString
createJsonFlavor JrpcConv,
requireAllFields = false
# JrpcConv is a namespace/flavor for encoding and decoding
# parameters and return value of a rpc method.

View File

@ -0,0 +1,306 @@
# json-rpc
# Copyright (c) 2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
std/hashes,
results,
json_serialization,
json_serialization/stew/results as jser_results
export
results,
json_serialization
# This module implements JSON-RPC 2.0 Specification
# https://www.jsonrpc.org/specification
type
# Special object of Json-RPC 2.0
JsonRPC2* = object
RequestParamKind* = enum
rpPositional
rpNamed
ParamDescRx* = object
kind* : JsonValueKind
param*: JsonString
ParamDescNamed* = object
name*: string
value*: JsonString
# Request params received by server
RequestParamsRx* = object
case kind*: RequestParamKind
of rpPositional:
positional*: seq[ParamDescRx]
of rpNamed:
named*: seq[ParamDescNamed]
# Request params sent by client
RequestParamsTx* = object
case kind*: RequestParamKind
of rpPositional:
positional*: seq[JsonString]
of rpNamed:
named*: seq[ParamDescNamed]
RequestIdKind* = enum
riNull
riNumber
riString
RequestId* = object
case kind*: RequestIdKind
of riNumber:
num*: int
of riString:
str*: string
of riNull:
discard
# Request received by server
RequestRx* = object
jsonrpc* : results.Opt[JsonRPC2]
id* : RequestId
`method`*: results.Opt[string]
params* : RequestParamsRx
# Request sent by client
RequestTx* = object
jsonrpc* : JsonRPC2
id* : results.Opt[RequestId]
`method`*: string
params* : RequestParamsTx
ResponseError* = object
code* : int
message*: string
data* : results.Opt[JsonString]
ResponseKind* = enum
rkResult
rkError
# Response sent by server
ResponseTx* = object
jsonrpc* : JsonRPC2
id* : RequestId
case kind*: ResponseKind
of rkResult:
result* : JsonString
of rkError:
error* : ResponseError
# Response received by client
ResponseRx* = object
jsonrpc*: results.Opt[JsonRPC2]
id* : results.Opt[RequestId]
result* : results.Opt[JsonString]
error* : results.Opt[ResponseError]
ReBatchKind* = enum
rbkSingle
rbkMany
RequestBatchRx* = object
case kind*: ReBatchKind
of rbkMany:
many* : seq[RequestRx]
of rbkSingle:
single*: RequestRx
RequestBatchTx* = object
case kind*: ReBatchKind
of rbkMany:
many* : seq[RequestTx]
of rbkSingle:
single*: RequestTx
ResponseBatchRx* = object
case kind*: ReBatchKind
of rbkMany:
many* : seq[ResponseRx]
of rbkSingle:
single*: ResponseRx
ResponseBatchTx* = object
case kind*: ReBatchKind
of rbkMany:
many* : seq[ResponseTx]
of rbkSingle:
single*: ResponseTx
# don't mix the json-rpc system encoding with the
# actual response/params encoding
createJsonFlavor JrpcSys,
requireAllFields = false
ResponseError.useDefaultSerializationIn JrpcSys
RequestTx.useDefaultWriterIn JrpcSys
ResponseRx.useDefaultReaderIn JrpcSys
RequestRx.useDefaultReaderIn JrpcSys
const
JsonRPC2Literal = JsonString("\"2.0\"")
{.push gcsafe, raises: [].}
func hash*(x: RequestId): hashes.Hash =
var h = 0.Hash
case x.kind:
of riNumber: h = h !& hash(x.num)
of riString: h = h !& hash(x.str)
of riNull: h = h !& hash("null")
result = !$(h)
func `$`*(x: RequestId): string =
case x.kind:
of riNumber: $x.num
of riString: x.str
of riNull: "null"
func `==`*(a, b: RequestId): bool =
if a.kind != b.kind:
return false
case a.kind
of riNumber: a.num == b.num
of riString: a.str == b.str
of riNull: true
func meth*(rx: RequestRx): Opt[string] =
rx.`method`
proc readValue*(r: var JsonReader[JrpcSys], val: var JsonRPC2)
{.gcsafe, raises: [IOError, JsonReaderError].} =
let version = r.parseAsString()
if version != JsonRPC2Literal:
r.raiseUnexpectedValue("Invalid JSON-RPC version, want=" &
JsonRPC2Literal.string & " got=" & version.string)
proc writeValue*(w: var JsonWriter[JrpcSys], val: JsonRPC2)
{.gcsafe, raises: [IOError].} =
w.writeValue JsonRPC2Literal
proc readValue*(r: var JsonReader[JrpcSys], val: var RequestId)
{.gcsafe, raises: [IOError, JsonReaderError].} =
let tok = r.tokKind
case tok
of JsonValueKind.Number:
val = RequestId(kind: riNumber, num: r.parseInt(int))
of JsonValueKind.String:
val = RequestId(kind: riString, str: r.parseString())
of JsonValueKind.Null:
val = RequestId(kind: riNull)
r.parseNull()
else:
r.raiseUnexpectedValue("Invalid RequestId, must be Number, String, or Null, got=" & $tok)
proc writeValue*(w: var JsonWriter[JrpcSys], val: RequestId)
{.gcsafe, raises: [IOError].} =
case val.kind
of riNumber: w.writeValue val.num
of riString: w.writeValue val.str
of riNull: w.writeValue JsonString("null")
proc readValue*(r: var JsonReader[JrpcSys], val: var RequestParamsRx)
{.gcsafe, raises: [IOError, SerializationError].} =
let tok = r.tokKind
case tok
of JsonValueKind.Array:
val = RequestParamsRx(kind: rpPositional)
r.parseArray:
val.positional.add ParamDescRx(
kind: r.tokKind(),
param: r.parseAsString(),
)
of JsonValueKind.Object:
val = RequestParamsRx(kind: rpNamed)
for key in r.readObjectFields():
val.named.add ParamDescNamed(
name: key,
value: r.parseAsString(),
)
else:
r.raiseUnexpectedValue("RequestParam must be either array or object, got=" & $tok)
proc writeValue*(w: var JsonWriter[JrpcSys], val: RequestParamsTx)
{.gcsafe, raises: [IOError].} =
case val.kind
of rpPositional:
w.writeArray val.positional
of rpNamed:
w.beginRecord RequestParamsTx
for x in val.named:
w.writeField(x.name, x.value)
w.endRecord()
proc writeValue*(w: var JsonWriter[JrpcSys], val: ResponseTx)
{.gcsafe, raises: [IOError].} =
w.beginRecord ResponseTx
w.writeField("jsonrpc", val.jsonrpc)
w.writeField("id", val.id)
if val.kind == rkResult:
w.writeField("result", val.result)
else:
w.writeField("error", val.error)
w.endRecord()
proc writeValue*(w: var JsonWriter[JrpcSys], val: RequestBatchTx)
{.gcsafe, raises: [IOError].} =
if val.kind == rbkMany:
w.writeArray(val.many)
else:
w.writeValue(val.single)
proc readValue*(r: var JsonReader[JrpcSys], val: var RequestBatchRx)
{.gcsafe, raises: [IOError, SerializationError].} =
let tok = r.tokKind
case tok
of JsonValueKind.Array:
val = RequestBatchRx(kind: rbkMany)
r.readValue(val.many)
of JsonValueKind.Object:
val = RequestBatchRx(kind: rbkSingle)
r.readValue(val.single)
else:
r.raiseUnexpectedValue("RequestBatch must be either array or object, got=" & $tok)
proc writeValue*(w: var JsonWriter[JrpcSys], val: ResponseBatchTx)
{.gcsafe, raises: [IOError].} =
if val.kind == rbkMany:
w.writeArray(val.many)
else:
w.writeValue(val.single)
proc readValue*(r: var JsonReader[JrpcSys], val: var ResponseBatchRx)
{.gcsafe, raises: [IOError, SerializationError].} =
let tok = r.tokKind
case tok
of JsonValueKind.Array:
val = ResponseBatchRx(kind: rbkMany)
r.readValue(val.many)
of JsonValueKind.Object:
val = ResponseBatchRx(kind: rbkSingle)
r.readValue(val.single)
else:
r.raiseUnexpectedValue("ResponseBatch must be either array or object, got=" & $tok)
proc toTx*(params: RequestParamsRx): RequestParamsTx =
case params.kind:
of rpPositional:
result = RequestParamsTx(kind: rpPositional)
for x in params.positional:
result.positional.add x.param
of rpNamed:
result = RequestParamsTx(kind: rpNamed)
result.named = params.named
{.pop.}

View File

@ -0,0 +1,301 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
std/[macros, typetraits],
stew/[byteutils, objects],
json_serialization,
json_serialization/std/[options],
./errors,
./jrpc_sys,
./jrpc_conv,
./shared_wrapper
export
jrpc_conv
{.push gcsafe, raises: [].}
proc unpackArg(args: JsonString, argName: string, argType: type): argType
{.gcsafe, raises: [JsonRpcError].} =
## This where input parameters are decoded from JSON into
## Nim data types
try:
result = JrpcConv.decode(args.string, argType)
except CatchableError as err:
raise newException(RequestDecodeError,
"Parameter [" & argName & "] of type '" &
$argType & "' could not be decoded: " & err.msg)
proc expectArrayLen(node, paramsIdent: NimNode, length: int) =
## Make sure positional params meets the handler expectation
let
expected = "Expected " & $length & " Json parameter(s) but got "
node.add quote do:
if `paramsIdent`.positional.len != `length`:
raise newException(RequestDecodeError, `expected` &
$`paramsIdent`.positional.len)
iterator paramsRevIter(params: NimNode): tuple[name, ntype: NimNode] =
## Bacward iterator of handler parameters
for i in countdown(params.len-1,1):
let arg = params[i]
let argType = arg[^2]
for j in 0 ..< arg.len-2:
yield (arg[j], argType)
proc isOptionalArg(typeNode: NimNode): bool =
typeNode.kind == nnkBracketExpr and
typeNode[0].kind == nnkIdent and
typeNode[0].strVal == "Option"
proc expectOptionalArrayLen(node: NimNode,
parameters: NimNode,
paramsIdent: NimNode,
maxLength: int): int =
## Validate if parameters sent by client meets
## minimum expectation of server
var minLength = maxLength
for arg, typ in paramsRevIter(parameters):
if not typ.isOptionalArg: break
dec minLength
let
expected = "Expected at least " & $minLength & " and maximum " &
$maxLength & " Json parameter(s) but got "
node.add quote do:
if `paramsIdent`.positional.len < `minLength`:
raise newException(RequestDecodeError, `expected` &
$`paramsIdent`.positional.len)
minLength
proc containsOptionalArg(params: NimNode): bool =
## Is one of handler parameters an optional?
for n, t in paramsIter(params):
if t.isOptionalArg:
return true
proc jsonToNim(paramVar: NimNode,
paramType: NimNode,
paramVal: NimNode,
paramName: string): NimNode =
## Convert a positional parameter from Json into Nim
result = quote do:
`paramVar` = `unpackArg`(`paramVal`, `paramName`, `paramType`)
proc calcActualParamCount(params: NimNode): int =
## this proc is needed to calculate the actual parameter count
## not matter what is the declaration form
## e.g. (a: U, b: V) vs. (a, b: T)
for n, t in paramsIter(params):
inc result
proc makeType(typeName, params: NimNode): NimNode =
## Generate type section contains an object definition
## with fields of handler params
let typeSec = quote do:
type `typeName` = object
let obj = typeSec[0][2]
let recList = newNimNode(nnkRecList)
if params.len > 1:
for i in 1..<params.len:
recList.add params[i]
obj[2] = recList
typeSec
proc setupPositional(params, paramsIdent: NimNode): (NimNode, int) =
## Generate code to check positional params length
var
minLength = 0
code = newStmtList()
if params.containsOptionalArg():
# more elaborate parameters array check
minLength = code.expectOptionalArrayLen(params, paramsIdent,
calcActualParamCount(params))
else:
# simple parameters array length check
code.expectArrayLen(paramsIdent, calcActualParamCount(params))
(code, minLength)
proc setupPositional(code: NimNode;
paramsObj, paramsIdent, paramIdent, paramType: NimNode;
pos, minLength: int) =
## processing multiple params of one type
## e.g. (a, b: T), including common (a: U, b: V) form
let
paramName = $paramIdent
paramVal = quote do:
`paramsIdent`.positional[`pos`].param
paramKind = quote do:
`paramsIdent`.positional[`pos`].kind
paramVar = quote do:
`paramsObj`.`paramIdent`
# e.g. (A: int, B: Option[int], C: string, D: Option[int], E: Option[string])
if paramType.isOptionalArg:
let
innerNode = jsonToNim(paramVar, paramType, paramVal, paramName)
if pos >= minLength:
# allow both empty and null after mandatory args
# D & E fall into this category
code.add quote do:
if `paramsIdent`.positional.len > `pos` and
`paramKind` != JsonValueKind.Null:
`innerNode`
else:
# allow null param for optional args between/before mandatory args
# B fall into this category
code.add quote do:
if `paramKind` != JsonValueKind.Null:
`innerNode`
else:
# mandatory args
# A and C fall into this category
# unpack Nim type and assign from json
code.add jsonToNim(paramVar, paramType, paramVal, paramName)
proc makeParams(retType: NimNode, params: NimNode): seq[NimNode] =
## Convert rpc params into handler params
result.add retType
if params.len > 1:
for i in 1..<params.len:
result.add params[i]
proc makeHandler(procName, params, procBody, returnInner: NimNode): NimNode =
## Generate rpc handler proc
let
returnType = quote do: Future[`returnInner`]
paramList = makeParams(returnType, params)
pragmas = quote do: {.async.}
result = newProc(
name = procName,
params = paramList,
body = procBody,
pragmas = pragmas
)
proc ofStmt(x, paramsObj, paramName, paramType: NimNode): NimNode =
let caseStr = $paramName
result = nnkOfBranch.newTree(
quote do: `caseStr`,
quote do:
`paramsObj`.`paramName` = unpackArg(`x`.value, `caseStr`, `paramType`)
)
proc setupNamed(paramsObj, paramsIdent, params: NimNode): NimNode =
let x = ident"x"
var caseStmt = nnkCaseStmt.newTree(
quote do: `x`.name
)
for paramName, paramType in paramsIter(params):
caseStmt.add ofStmt(x, paramsObj, paramName, paramType)
caseStmt.add nnkElse.newTree(
quote do: discard
)
result = quote do:
for `x` in `paramsIdent`.named:
`caseStmt`
proc wrapServerHandler*(methName: string, params, procBody, procWrapper: NimNode): NimNode =
## This proc generate something like this:
##
## proc rpcHandler(paramA: ParamAType, paramB: ParamBType): Future[ReturnType] =
## procBody
## return retVal
##
## proc rpcWrapper(params: RequestParamsRx): Future[StringOfJson] =
## type
## RpcType = object
## paramA: ParamAType
## paramB: ParamBType
##
## var rpcVar: RpcType
##
## if params.isPositional:
## if params.positional.len < expectedLen:
## raise exception
## rpcVar.paramA = params.unpack(paramA of ParamAType)
## rpcVar.paramB = params.unpack(paramB of ParamBType)
## else:
## rpcVar = params.unpack(named of RpcType)
##
## let res = await rpcHandler(rpcVar.paramA, rpcVar.paramB)
## return JrpcConv.encode(res).StringOfJson
let
params = params.ensureReturnType()
setup = newStmtList()
typeName = genSym(nskType, "RpcType")
paramsObj = ident"rpcVar"
handlerName = genSym(nskProc, methName & "_rpcHandler")
paramsIdent = genSym(nskParam, "rpcParams")
returnType = params[0]
hasParams = params.len > 1 # not including return type
(posSetup, minLength) = setupPositional(params, paramsIdent)
handler = makeHandler(handlerName, params, procBody, returnType)
named = setupNamed(paramsObj, paramsIdent, params)
if hasParams:
setup.add makeType(typeName, params)
setup.add quote do:
var `paramsObj`: `typeName`
# unpack each parameter and provide assignments
var
pos = 0
positional = newStmtList()
executeParams: seq[NimNode]
for paramIdent, paramType in paramsIter(params):
positional.setupPositional(paramsObj, paramsIdent,
paramIdent, paramType, pos, minLength)
executeParams.add quote do:
`paramsObj`.`paramIdent`
inc pos
if hasParams:
setup.add quote do:
if `paramsIdent`.kind == rpPositional:
`posSetup`
`positional`
else:
`named`
else:
setup.add quote do:
if `paramsIdent`.kind == rpPositional:
`posSetup`
let
awaitedResult = ident "awaitedResult"
doEncode = quote do: encode(JrpcConv, `awaitedResult`)
maybeWrap =
if returnType.noWrap: awaitedResult
else: ident"StringOfJson".newCall doEncode
executeCall = newCall(handlerName, executeParams)
result = newStmtList()
result.add handler
result.add quote do:
proc `procWrapper`(`paramsIdent`: RequestParamsRx): Future[StringOfJson] {.async, gcsafe.} =
# Avoid 'yield in expr not lowered' with an intermediate variable.
# See: https://github.com/nim-lang/Nim/issues/17849
`setup`
let `awaitedResult` = await `executeCall`
return `maybeWrap`

View File

@ -0,0 +1,70 @@
# json-rpc
# Copyright (c) 2024 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
std/[json, macros],
./jrpc_sys,
./jrpc_conv
iterator paramsIter*(params: NimNode): tuple[name, ntype: NimNode] =
## Forward iterator of handler parameters
for i in 1 ..< params.len:
let arg = params[i]
let argType = arg[^2]
for j in 0 ..< arg.len-2:
yield (arg[j], argType)
func ensureReturnType*(params: NimNode): NimNode =
let retType = ident"JsonNode"
if params.isNil or params.kind == nnkEmpty or params.len == 0:
return nnkFormalParams.newTree(retType)
if params.len >= 1 and params[0].kind == nnkEmpty:
params[0] = retType
params
func noWrap*(returnType: NimNode): bool =
## Condition when return type should not be encoded
## to Json
returnType.repr == "StringOfJson" or
returnType.repr == "JsonString"
func paramsTx*(params: JsonNode): RequestParamsTx =
if params.kind == JArray:
var args: seq[JsonString]
for x in params:
args.add JrpcConv.encode(x).JsonString
RequestParamsTx(
kind: rpPositional,
positional: system.move(args),
)
elif params.kind == JObject:
var args: seq[ParamDescNamed]
for k, v in params:
args.add ParamDescNamed(
name: k,
value: JrpcConv.encode(v).JsonString,
)
RequestParamsTx(
kind: rpNamed,
named: system.move(args),
)
else:
RequestParamsTx(
kind: rpPositional,
positional: @[JrpcConv.encode(params).JsonString],
)
func requestTx*(name: string, params: RequestParamsTx, id: RequestId): RequestTx =
RequestTx(
id: Opt.some(id),
`method`: name,
params: params,
)

View File

@ -8,135 +8,204 @@
# those terms. # those terms.
import import
std/[macros, strutils, tables], std/[macros, tables, json],
chronicles, chronos, json_serialization/writer, chronicles,
./jsonmarshal, ./errors chronos,
./private/server_handler_wrapper,
./private/errors,
./private/jrpc_sys
export export
chronos, jsonmarshal chronos,
jrpc_conv,
json
type type
StringOfJson* = JsonString
# Procedure signature accepted as an RPC call by server # Procedure signature accepted as an RPC call by server
RpcProc* = proc(input: JsonNode): Future[StringOfJson] {.gcsafe, raises: [Defect].} RpcProc* = proc(params: RequestParamsRx): Future[StringOfJson]
{.gcsafe, raises: [CatchableError].}
RpcRouter* = object RpcRouter* = object
procs*: Table[string, RpcProc] procs*: Table[string, RpcProc]
const const
methodField = "method"
paramsField = "params"
JSON_PARSE_ERROR* = -32700 JSON_PARSE_ERROR* = -32700
INVALID_REQUEST* = -32600 INVALID_REQUEST* = -32600
METHOD_NOT_FOUND* = -32601 METHOD_NOT_FOUND* = -32601
INVALID_PARAMS* = -32602 INVALID_PARAMS* = -32602
INTERNAL_ERROR* = -32603 INTERNAL_ERROR* = -32603
SERVER_ERROR* = -32000 SERVER_ERROR* = -32000
JSON_ENCODE_ERROR* = -32001
defaultMaxRequestLength* = 1024 * 128 defaultMaxRequestLength* = 1024 * 128
{.push gcsafe, raises: [].}
# ------------------------------------------------------------------------------
# Private helpers
# ------------------------------------------------------------------------------
func invalidRequest(msg: string): ResponseError =
ResponseError(code: INVALID_REQUEST, message: msg)
func methodNotFound(msg: string): ResponseError =
ResponseError(code: METHOD_NOT_FOUND, message: msg)
func serverError(msg: string, data: StringOfJson): ResponseError =
ResponseError(code: SERVER_ERROR, message: msg, data: Opt.some(data))
func somethingError(code: int, msg: string): ResponseError =
ResponseError(code: code, message: msg)
proc validateRequest(router: RpcRouter, req: RequestRx):
Result[RpcProc, ResponseError] =
if req.jsonrpc.isNone:
return invalidRequest("'jsonrpc' missing or invalid").err
if req.id.kind == riNull:
return invalidRequest("'id' missing or invalid").err
if req.meth.isNone:
return invalidRequest("'method' missing or invalid").err
let
methodName = req.meth.get
rpcProc = router.procs.getOrDefault(methodName)
if rpcProc.isNil:
return methodNotFound("'" & methodName &
"' is not a registered RPC method").err
ok(rpcProc)
proc wrapError(err: ResponseError, id: RequestId): ResponseTx =
ResponseTx(
id: id,
kind: rkError,
error: err,
)
proc wrapError(code: int, msg: string, id: RequestId): ResponseTx =
ResponseTx(
id: id,
kind: rkError,
error: somethingError(code, msg),
)
proc wrapReply(res: StringOfJson, id: RequestId): ResponseTx =
ResponseTx(
id: id,
kind: rkResult,
result: res,
)
proc wrapError(code: int, msg: string): string =
"""{"jsonrpc":"2.0","id":null,"error":{"code":""" & $code &
""","message":""" & escapeJson(msg) & "}}"
# ------------------------------------------------------------------------------
# Public functions
# ------------------------------------------------------------------------------
proc init*(T: type RpcRouter): T = discard proc init*(T: type RpcRouter): T = discard
proc newRpcRouter*: RpcRouter {.deprecated.} = proc register*(router: var RpcRouter, path: string, call: RpcProc)
RpcRouter.init() {.gcsafe, raises: [CatchableError].} =
proc register*(router: var RpcRouter, path: string, call: RpcProc) =
router.procs[path] = call router.procs[path] = call
proc clear*(router: var RpcRouter) = proc clear*(router: var RpcRouter) =
router.procs.clear router.procs.clear
proc hasMethod*(router: RpcRouter, methodName: string): bool = router.procs.hasKey(methodName) proc hasMethod*(router: RpcRouter, methodName: string): bool =
router.procs.hasKey(methodName)
func isEmpty(node: JsonNode): bool = node.isNil or node.kind == JNull proc route*(router: RpcRouter, req: RequestRx):
Future[ResponseTx] {.gcsafe, async: (raises: []).} =
let rpcProc = router.validateRequest(req).valueOr:
return wrapError(error, req.id)
# Json reply wrappers try:
let res = await rpcProc(req.params)
return wrapReply(res, req.id)
except InvalidRequest as err:
return wrapError(err.code, err.msg, req.id)
except CatchableError as err:
let methodName = req.meth.get # this Opt already validated
debug "Error occurred within RPC",
methodName = methodName, err = err.msg
return serverError(methodName & " raised an exception",
escapeJson(err.msg).StringOfJson).
wrapError(req.id)
# https://www.jsonrpc.org/specification#response_object proc wrapErrorAsync*(code: int, msg: string):
proc wrapReply*(id: JsonNode, value: StringOfJson): StringOfJson = Future[StringOfJson] {.gcsafe, async: (raises: []).} =
# Success response carries version, id and result fields only return wrapError(code, msg).StringOfJson
StringOfJson(
"""{"jsonrpc":"2.0","id":$1,"result":$2}""" % [$id, string(value)] & "\r\n")
proc wrapError*(code: int, msg: string, id: JsonNode = newJNull(), proc route*(router: RpcRouter, data: string):
data: JsonNode = newJNull()): StringOfJson = Future[string] {.gcsafe, async: (raises: []).} =
# Error reply that carries version, id and error object only ## Route to RPC from string data. Data is expected to be able to be
StringOfJson( ## converted to Json.
"""{"jsonrpc":"2.0","id":$1,"error":{"code":$2,"message":$3,"data":$4}}""" % [
$id, $code, escapeJson(msg), $data
] & "\r\n")
proc route*(router: RpcRouter, node: JsonNode): Future[StringOfJson] {.async, gcsafe.} =
if node{"jsonrpc"}.getStr() != "2.0":
return wrapError(INVALID_REQUEST, "'jsonrpc' missing or invalid")
let id = node{"id"}
if id == nil:
return wrapError(INVALID_REQUEST, "'id' missing or invalid")
let methodName = node{"method"}.getStr()
if methodName.len == 0:
return wrapError(INVALID_REQUEST, "'method' missing or invalid")
let rpcProc = router.procs.getOrDefault(methodName)
let params = node.getOrDefault("params")
if rpcProc == nil:
return wrapError(METHOD_NOT_FOUND, "'" & methodName & "' is not a registered RPC method", id)
else:
try:
let res = await rpcProc(if params == nil: newJArray() else: params)
return wrapReply(id, res)
except InvalidRequest as err:
return wrapError(err.code, err.msg, id)
except CatchableError as err:
debug "Error occurred within RPC", methodName = methodName, err = err.msg
return wrapError(
SERVER_ERROR, methodName & " raised an exception", id, newJString(err.msg))
proc route*(router: RpcRouter, data: string): Future[string] {.async, gcsafe.} =
## Route to RPC from string data. Data is expected to be able to be converted to Json.
## Returns string of Json from RPC result/error node ## Returns string of Json from RPC result/error node
when defined(nimHasWarnBareExcept): when defined(nimHasWarnBareExcept):
{.warning[BareExcept]:off.} {.warning[BareExcept]:off.}
let node = let request =
try: parseJson(data) try:
JrpcSys.decode(data, RequestRx)
except CatchableError as err: except CatchableError as err:
return string(wrapError(JSON_PARSE_ERROR, err.msg)) return wrapError(JSON_PARSE_ERROR, err.msg)
except Exception as err: except Exception as err:
# TODO https://github.com/status-im/nimbus-eth2/issues/2430 # TODO https://github.com/status-im/nimbus-eth2/issues/2430
return string(wrapError(JSON_PARSE_ERROR, err.msg)) return wrapError(JSON_PARSE_ERROR, err.msg)
let reply =
try:
let response = await router.route(request)
JrpcSys.encode(response)
except CatchableError as err:
return wrapError(JSON_ENCODE_ERROR, err.msg)
except Exception as err:
return wrapError(JSON_ENCODE_ERROR, err.msg)
when defined(nimHasWarnBareExcept): when defined(nimHasWarnBareExcept):
{.warning[BareExcept]:on.} {.warning[BareExcept]:on.}
return string(await router.route(node)) return reply
proc tryRoute*(router: RpcRouter, data: JsonNode, fut: var Future[StringOfJson]): bool = proc tryRoute*(router: RpcRouter, data: StringOfJson,
fut: var Future[StringOfJson]): Result[void, string] =
## Route to RPC, returns false if the method or params cannot be found. ## Route to RPC, returns false if the method or params cannot be found.
## Expects json input and returns json output. ## Expects json input and returns json output.
let when defined(nimHasWarnBareExcept):
jPath = data.getOrDefault(methodField) {.warning[BareExcept]:off.}
jParams = data.getOrDefault(paramsField) {.warning[UnreachableCode]:off.}
if jPath.isEmpty or jParams.isEmpty:
return false
let try:
path = jPath.getStr let req = JrpcSys.decode(data.string, RequestRx)
rpc = router.procs.getOrDefault(path)
if rpc != nil:
fut = rpc(jParams)
return true
proc hasReturnType(params: NimNode): bool = if req.jsonrpc.isNone:
if params != nil and params.len > 0 and params[0] != nil and return err("`jsonrpc` missing or invalid")
params[0].kind != nnkEmpty:
result = true
macro rpc*(server: RpcRouter, path: string, body: untyped): untyped = if req.meth.isNone:
return err("`method` missing or invalid")
let rpc = router.procs.getOrDefault(req.meth.get)
if rpc.isNil:
return err("rpc method not found: " & req.meth.get)
fut = rpc(req.params)
return ok()
except CatchableError as ex:
return err(ex.msg)
except Exception as ex:
return err(ex.msg)
when defined(nimHasWarnBareExcept):
{.warning[BareExcept]:on.}
{.warning[UnreachableCode]:on.}
macro rpc*(server: RpcRouter, path: static[string], body: untyped): untyped =
## Define a remote procedure call. ## Define a remote procedure call.
## Input and return parameters are defined using the ``do`` notation. ## Input and return parameters are defined using the ``do`` notation.
## For example: ## For example:
@ -146,41 +215,17 @@ macro rpc*(server: RpcRouter, path: string, body: untyped): untyped =
## ``` ## ```
## Input parameters are automatically marshalled from json to Nim types, ## Input parameters are automatically marshalled from json to Nim types,
## and output parameters are automatically marshalled to json for transport. ## and output parameters are automatically marshalled to json for transport.
result = newStmtList()
let let
parameters = body.findChild(it.kind == nnkFormalParams) params = body.findChild(it.kind == nnkFormalParams)
# all remote calls have a single parameter: `params: JsonNode`
paramsIdent = newIdentNode"params"
rpcProcImpl = genSym(nskProc)
rpcProcWrapper = genSym(nskProc)
var
setup = jsonToNim(parameters, paramsIdent)
procBody = if body.kind == nnkStmtList: body else: body.body procBody = if body.kind == nnkStmtList: body else: body.body
procWrapper = genSym(nskProc, $path & "_rpcWrapper")
let ReturnType = if parameters.hasReturnType: parameters[0] result = wrapServerHandler($path, params, procBody, procWrapper)
else: ident "JsonNode"
# delegate async proc allows return and setting of result as native type
result.add quote do:
proc `rpcProcImpl`(`paramsIdent`: JsonNode): Future[`ReturnType`] {.async.} =
`setup`
`procBody`
let
awaitedResult = ident "awaitedResult"
doEncode = quote do: encode(JsonRpc, `awaitedResult`)
maybeWrap =
if ReturnType == ident"StringOfJson": doEncode
else: ident"StringOfJson".newCall doEncode
result.add quote do: result.add quote do:
proc `rpcProcWrapper`(`paramsIdent`: JsonNode): Future[StringOfJson] {.async, gcsafe.} = `server`.register(`path`, `procWrapper`)
# Avoid 'yield in expr not lowered' with an intermediate variable.
# See: https://github.com/nim-lang/Nim/issues/17849
let `awaitedResult` = await `rpcProcImpl`(`paramsIdent`)
return `maybeWrap`
`server`.register(`path`, `rpcProcWrapper`)
when defined(nimDumpRpcs): when defined(nimDumpRpcs):
echo "\n", path, ": ", result.repr echo "\n", path, ": ", result.repr
{.pop.}

View File

@ -7,12 +7,11 @@
# This file may not be copied, modified, or distributed except according to # This file may not be copied, modified, or distributed except according to
# those terms. # those terms.
{.push raises: [Defect].}
import import
pkg/websock/websock, pkg/websock/websock,
./servers/[httpserver], ./servers/[httpserver],
./clients/[httpclient, websocketclient] ./clients/[httpclient, websocketclient],
./private/jrpc_sys
type type
ClientKind* = enum ClientKind* = enum
@ -40,6 +39,8 @@ type
compression*: bool compression*: bool
flags*: set[TLSFlags] flags*: set[TLSFlags]
{.push gcsafe, raises: [].}
# TODO Add validations that provided uri-s are correct https/wss uri and retrun # TODO Add validations that provided uri-s are correct https/wss uri and retrun
# Result[string, ClientConfig] # Result[string, ClientConfig]
proc getHttpClientConfig*(uri: string): ClientConfig = proc getHttpClientConfig*(uri: string): ClientConfig =
@ -53,9 +54,9 @@ proc getWebSocketClientConfig*(
ClientConfig(kind: WebSocket, wsUri: uri, compression: compression, flags: flags) ClientConfig(kind: WebSocket, wsUri: uri, compression: compression, flags: flags)
proc proxyCall(client: RpcClient, name: string): RpcProc = proc proxyCall(client: RpcClient, name: string): RpcProc =
return proc (params: JsonNode): Future[StringOfJson] {.async.} = return proc (params: RequestParamsRx): Future[StringOfJson] {.gcsafe, async.} =
let res = await client.call(name, params) let res = await client.call(name, params.toTx)
return StringOfJson($res) return res
proc getClient*(proxy: RpcProxy): RpcClient = proc getClient*(proxy: RpcProxy): RpcClient =
case proxy.kind case proxy.kind
@ -85,14 +86,14 @@ proc new*(
listenAddresses: openArray[TransportAddress], listenAddresses: openArray[TransportAddress],
cfg: ClientConfig, cfg: ClientConfig,
authHooks: seq[HttpAuthHook] = @[] authHooks: seq[HttpAuthHook] = @[]
): T {.raises: [Defect, CatchableError].} = ): T {.raises: [CatchableError].} =
RpcProxy.new(newRpcHttpServer(listenAddresses, RpcRouter.init(), authHooks), cfg) RpcProxy.new(newRpcHttpServer(listenAddresses, RpcRouter.init(), authHooks), cfg)
proc new*( proc new*(
T: type RpcProxy, T: type RpcProxy,
listenAddresses: openArray[string], listenAddresses: openArray[string],
cfg: ClientConfig, cfg: ClientConfig,
authHooks: seq[HttpAuthHook] = @[]): T {.raises: [Defect, CatchableError].} = authHooks: seq[HttpAuthHook] = @[]): T {.raises: [CatchableError].} =
RpcProxy.new(newRpcHttpServer(listenAddresses, RpcRouter.init(), authHooks), cfg) RpcProxy.new(newRpcHttpServer(listenAddresses, RpcRouter.init(), authHooks), cfg)
proc connectToProxy(proxy: RpcProxy): Future[void] = proc connectToProxy(proxy: RpcProxy): Future[void] =
@ -125,3 +126,5 @@ proc stop*(proxy: RpcProxy) {.async.} =
proc closeWait*(proxy: RpcProxy) {.async.} = proc closeWait*(proxy: RpcProxy) {.async.} =
await proxy.rpcHttpServer.closeWait() await proxy.rpcHttpServer.closeWait()
{.pop.}

View File

@ -8,21 +8,35 @@
# those terms. # those terms.
import import
std/tables, std/json,
chronos, chronos,
./router, ./router,
./jsonmarshal ./private/jrpc_conv,
./private/jrpc_sys,
./private/shared_wrapper,
./private/errors
export chronos, jsonmarshal, router export
chronos,
jrpc_conv,
router
type type
RpcServer* = ref object of RootRef RpcServer* = ref object of RootRef
router*: RpcRouter router*: RpcRouter
proc new(T: type RpcServer): T = {.push gcsafe, raises: [].}
# ------------------------------------------------------------------------------
# Constructors
# ------------------------------------------------------------------------------
proc new*(T: type RpcServer): T =
T(router: RpcRouter.init()) T(router: RpcRouter.init())
proc newRpcServer*(): RpcServer {.deprecated.} = RpcServer.new() # ------------------------------------------------------------------------------
# Public functions
# ------------------------------------------------------------------------------
template rpc*(server: RpcServer, path: string, body: untyped): untyped = template rpc*(server: RpcServer, path: string, body: untyped): untyped =
server.router.rpc(path, body) server.router.rpc(path, body)
@ -32,8 +46,23 @@ template hasMethod*(server: RpcServer, methodName: string): bool =
proc executeMethod*(server: RpcServer, proc executeMethod*(server: RpcServer,
methodName: string, methodName: string,
args: JsonNode): Future[StringOfJson] = params: RequestParamsTx): Future[StringOfJson]
server.router.procs[methodName](args) {.gcsafe, raises: [JsonRpcError].} =
let
req = requestTx(methodName, params, RequestId(kind: riNumber, num: 0))
reqData = JrpcSys.encode(req).JsonString
server.router.tryRoute(reqData, result).isOkOr:
raise newException(JsonRpcError, error)
proc executeMethod*(server: RpcServer,
methodName: string,
args: JsonNode): Future[StringOfJson]
{.gcsafe, raises: [JsonRpcError].} =
let params = paramsTx(args)
server.executeMethod(methodName, params)
# Wrapper for message processing # Wrapper for message processing
@ -42,10 +71,12 @@ proc route*(server: RpcServer, line: string): Future[string] {.gcsafe.} =
# Server registration # Server registration
proc register*(server: RpcServer, name: string, rpc: RpcProc) = proc register*(server: RpcServer, name: string, rpc: RpcProc) {.gcsafe, raises: [CatchableError].} =
## Add a name/code pair to the RPC server. ## Add a name/code pair to the RPC server.
server.router.register(name, rpc) server.router.register(name, rpc)
proc unRegisterAll*(server: RpcServer) = proc unRegisterAll*(server: RpcServer) =
# Remove all remote procedure calls from this server. # Remove all remote procedure calls from this server.
server.router.clear server.router.clear
{.pop.}

View File

@ -11,9 +11,11 @@ import
stew/byteutils, stew/byteutils,
chronicles, httputils, chronos, chronicles, httputils, chronos,
chronos/apps/http/[httpserver, shttpserver], chronos/apps/http/[httpserver, shttpserver],
".."/[errors, server] ../private/errors,
../server
export server, shttpserver export
server, shttpserver
logScope: logScope:
topics = "JSONRPC-HTTP-SERVER" topics = "JSONRPC-HTTP-SERVER"
@ -36,41 +38,52 @@ type
httpServers: seq[HttpServerRef] httpServers: seq[HttpServerRef]
authHooks: seq[HttpAuthHook] authHooks: seq[HttpAuthHook]
proc processClientRpc(rpcServer: RpcHttpServer): HttpProcessCallback = proc processClientRpc(rpcServer: RpcHttpServer): HttpProcessCallback2 =
return proc (req: RequestFence): Future[HttpResponseRef] {.async.} = return proc (req: RequestFence): Future[HttpResponseRef] {.async: (raises: [CancelledError]).} =
if req.isOk(): if not req.isOk():
let request = req.get() return defaultResponse()
# if hook result is not nil, let request = req.get()
# it means we should return immediately # if hook result is not nil,
# it means we should return immediately
try:
for hook in rpcServer.authHooks: for hook in rpcServer.authHooks:
let res = await hook(request) let res = await hook(request)
if not res.isNil: if not res.isNil:
return res return res
except CatchableError as exc:
error "Internal error while processing JSON-RPC hook", msg=exc.msg
try:
return await request.respond(
Http503,
"Internal error while processing JSON-RPC hook: " & exc.msg)
except HttpWriteError as exc:
error "Something error", msg=exc.msg
return defaultResponse()
let
headers = HttpTable.init([("Content-Type",
"application/json; charset=utf-8")])
try:
let let
body = await request.getBody() body = await request.getBody()
headers = HttpTable.init([("Content-Type",
"application/json; charset=utf-8")])
data =
try:
await rpcServer.route(string.fromBytes(body))
except CancelledError as exc:
raise exc
except CatchableError as exc:
debug "Internal error while processing JSON-RPC call"
return await request.respond(
Http503,
"Internal error while processing JSON-RPC call: " & exc.msg,
headers)
data = await rpcServer.route(string.fromBytes(body))
res = await request.respond(Http200, data, headers) res = await request.respond(Http200, data, headers)
trace "JSON-RPC result has been sent" trace "JSON-RPC result has been sent"
return res return res
else: except CancelledError as exc:
return dumbResponse() raise exc
except CatchableError as exc:
debug "Internal error while processing JSON-RPC call"
try:
return await request.respond(
Http503,
"Internal error while processing JSON-RPC call: " & exc.msg)
except HttpWriteError as exc:
error "Something error", msg=exc.msg
return defaultResponse()
proc addHttpServer*( proc addHttpServer*(
rpcServer: RpcHttpServer, rpcServer: RpcHttpServer,

View File

@ -10,7 +10,8 @@
import import
chronicles, chronicles,
json_serialization/std/net, json_serialization/std/net,
".."/[errors, server] ../private/errors,
../server
export errors, server export errors, server
@ -18,26 +19,25 @@ type
RpcSocketServer* = ref object of RpcServer RpcSocketServer* = ref object of RpcServer
servers: seq[StreamServer] servers: seq[StreamServer]
proc sendError*[T](transport: T, code: int, msg: string, id: JsonNode, proc processClient(server: StreamServer, transport: StreamTransport) {.async: (raises: []), gcsafe.} =
data: JsonNode = newJNull()) {.async.} =
## Send error message to client
let error = wrapError(code, msg, id, data)
result = transport.write(string wrapReply(id, StringOfJson("null"), error))
proc processClient(server: StreamServer, transport: StreamTransport) {.async, gcsafe.} =
## Process transport data to the RPC server ## Process transport data to the RPC server
var rpc = getUserData[RpcSocketServer](server) try:
while true: var rpc = getUserData[RpcSocketServer](server)
var while true:
value = await transport.readLine(defaultMaxRequestLength) var
if value == "": value = await transport.readLine(defaultMaxRequestLength)
await transport.closeWait() if value == "":
break await transport.closeWait()
break
debug "Processing message", address = transport.remoteAddress(), line = value debug "Processing message", address = transport.remoteAddress(), line = value
let res = await rpc.route(value) let res = await rpc.route(value)
discard await transport.write(res) discard await transport.write(res & "\r\n")
except TransportError as ex:
error "Transport closed during processing client", msg=ex.msg
except CatchableError as ex:
error "Error occured during processing client", msg=ex.msg
# Utility functions for setting up servers using stream transport addresses # Utility functions for setting up servers using stream transport addresses

View File

@ -1,3 +1,12 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
{. warning[UnusedImport]:off .} {. warning[UnusedImport]:off .}
import import
@ -6,4 +15,7 @@ import
testhttp, testhttp,
testserverclient, testserverclient,
testproxy, testproxy,
testhook testhook,
test_jrpc_sys,
test_router_rpc,
test_callsigs

View File

@ -1,11 +0,0 @@
import
../json_rpc/router
converter toStr*(value: distinct (string|StringOfJson)): string = string(value)
template `==`*(a: StringOfJson, b: JsonNode): bool =
parseJson(string a) == b
template `==`*(a: JsonNode, b: StringOfJson): bool =
a == parseJson(string b)

View File

@ -41,23 +41,14 @@ proc eth_getCompilers(): seq[string]
proc eth_compileLLL(): seq[byte] proc eth_compileLLL(): seq[byte]
proc eth_compileSolidity(): seq[byte] proc eth_compileSolidity(): seq[byte]
proc eth_compileSerpent(): seq[byte] proc eth_compileSerpent(): seq[byte]
proc eth_newFilter(filterOptions: FilterOptions): int
proc eth_newBlockFilter(): int proc eth_newBlockFilter(): int
proc eth_newPendingTransactionFilter(): int proc eth_newPendingTransactionFilter(): int
proc eth_uninstallFilter(filterId: int): bool proc eth_uninstallFilter(filterId: int): bool
proc eth_getFilterChanges(filterId: int): seq[LogObject]
proc eth_getFilterLogs(filterId: int): seq[LogObject]
proc eth_getLogs(filterOptions: FilterOptions): seq[LogObject]
proc eth_getWork(): seq[UInt256] proc eth_getWork(): seq[UInt256]
proc eth_submitWork(nonce: int64, powHash: Uint256, mixDigest: Uint256): bool proc eth_submitWork(nonce: int64, powHash: Uint256, mixDigest: Uint256): bool
proc eth_submitHashrate(hashRate: UInt256, id: Uint256): bool proc eth_submitHashrate(hashRate: UInt256, id: Uint256): bool
proc shh_post(): string proc shh_post(): string
proc shh_version(message: WhisperPost): bool
proc shh_newIdentity(): array[60, byte] proc shh_newIdentity(): array[60, byte]
proc shh_hasIdentity(identity: array[60, byte]): bool proc shh_hasIdentity(identity: array[60, byte]): bool
proc shh_newGroup(): array[60, byte] proc shh_newGroup(): array[60, byte]
proc shh_addToGroup(identity: array[60, byte]): bool
proc shh_newFilter(filterOptions: FilterOptions, to: array[60, byte], topics: seq[UInt256]): int
proc shh_uninstallFilter(id: int): bool proc shh_uninstallFilter(id: int): bool
proc shh_getFilterChanges(id: int): seq[WhisperMessage]
proc shh_getMessages(id: int): seq[WhisperMessage]

View File

@ -1,9 +1,14 @@
import
../../json_rpc/private/errors
type type
HexQuantityStr* = distinct string HexQuantityStr* = distinct string
HexDataStr* = distinct string HexDataStr* = distinct string
# Hex validation # Hex validation
{.push gcsafe, raises: [].}
template stripLeadingZeros(value: string): string = template stripLeadingZeros(value: string): string =
var cidx = 0 var cidx = 0
# ignore the last character so we retain '0' on zero value # ignore the last character so we retain '0' on zero value
@ -61,53 +66,55 @@ template hexQuantityStr*(value: string): HexQuantityStr = value.HexQuantityStr
# Converters # Converters
import json import json
import ../json_rpc/jsonmarshal import ../../json_rpc/private/jrpc_conv
proc `%`*(value: HexDataStr): JsonNode = proc `%`*(value: HexDataStr): JsonNode {.gcsafe, raises: [JsonRpcError].} =
if not value.validate: if not value.validate:
raise newException(ValueError, "HexDataStr: Invalid hex for Ethereum: " & value.string) raise newException(JsonRpcError, "HexDataStr: Invalid hex for Ethereum: " & value.string)
else: else:
result = %(value.string) result = %(value.string)
proc `%`*(value: HexQuantityStr): JsonNode = proc `%`*(value: HexQuantityStr): JsonNode {.gcsafe, raises: [JsonRpcError].} =
if not value.validate: if not value.validate:
raise newException(ValueError, "HexQuantityStr: Invalid hex for Ethereum: " & value.string) raise newException(JsonRpcError, "HexQuantityStr: Invalid hex for Ethereum: " & value.string)
else: else:
result = %(value.string) result = %(value.string)
proc writeValue*(w: var JsonWriter[JsonRpc], val: HexDataStr) {.raises: [IOError].} = proc writeValue*(w: var JsonWriter[JrpcConv], val: HexDataStr) {.raises: [IOError].} =
writeValue(w, val.string) writeValue(w, val.string)
proc writeValue*(w: var JsonWriter[JsonRpc], val: HexQuantityStr) {.raises: [IOError].} = proc writeValue*(w: var JsonWriter[JrpcConv], val: HexQuantityStr) {.raises: [IOError].} =
writeValue(w, $val.string) writeValue(w, $val.string)
proc readValue*(r: var JsonReader[JsonRpc], v: var HexDataStr) = proc readValue*(r: var JsonReader[JrpcConv], v: var HexDataStr) {.gcsafe, raises: [JsonReaderError].} =
# Note that '0x' is stripped after validation # Note that '0x' is stripped after validation
try: try:
let hexStr = readValue(r, string) let hexStr = readValue(r, string)
if not hexStr.hexDataStr.validate: if not hexStr.hexDataStr.validate:
raise newException(ValueError, "Value for '" & $v.type & "' is not valid as a Ethereum data \"" & hexStr & "\"") raise newException(JsonRpcError, "Value for '" & $v.type & "' is not valid as a Ethereum data \"" & hexStr & "\"")
v = hexStr[2..hexStr.high].hexDataStr v = hexStr[2..hexStr.high].hexDataStr
except Exception as err: except Exception as err:
r.raiseUnexpectedValue("Error deserializing for '" & $v.type & "' stream: " & err.msg) r.raiseUnexpectedValue("Error deserializing for '" & $v.type & "' stream: " & err.msg)
proc readValue*(r: var JsonReader[JsonRpc], v: var HexQuantityStr) = proc readValue*(r: var JsonReader[JrpcConv], v: var HexQuantityStr) {.gcsafe, raises: [JsonReaderError].} =
# Note that '0x' is stripped after validation # Note that '0x' is stripped after validation
try: try:
let hexStr = readValue(r, string) let hexStr = readValue(r, string)
if not hexStr.hexQuantityStr.validate: if not hexStr.hexQuantityStr.validate:
raise newException(ValueError, "Value for '" & $v.type & "' is not valid as a Ethereum data \"" & hexStr & "\"") raise newException(JsonRpcError, "Value for '" & $v.type & "' is not valid as a Ethereum data \"" & hexStr & "\"")
v = hexStr[2..hexStr.high].hexQuantityStr v = hexStr[2..hexStr.high].hexQuantityStr
except Exception as err: except Exception as err:
r.raiseUnexpectedValue("Error deserializing for '" & $v.type & "' stream: " & err.msg) r.raiseUnexpectedValue("Error deserializing for '" & $v.type & "' stream: " & err.msg)
{.pop.}
# testing # testing
when isMainModule: when isMainModule:
import unittest import unittest
suite "Hex quantity": suite "Hex quantity":
test "Empty string": test "Empty string":
expect ValueError: expect JsonRpcError:
let let
source = "" source = ""
x = hexQuantityStr source x = hexQuantityStr source
@ -123,17 +130,17 @@ when isMainModule:
x = hexQuantityStr"0x123" x = hexQuantityStr"0x123"
check %x == %source check %x == %source
test "Missing header": test "Missing header":
expect ValueError: expect JsonRpcError:
let let
source = "1234" source = "1234"
x = hexQuantityStr source x = hexQuantityStr source
check %x != %source check %x != %source
expect ValueError: expect JsonRpcError:
let let
source = "01234" source = "01234"
x = hexQuantityStr source x = hexQuantityStr source
check %x != %source check %x != %source
expect ValueError: expect JsonRpcError:
let let
source = "x1234" source = "x1234"
x = hexQuantityStr source x = hexQuantityStr source
@ -146,23 +153,23 @@ when isMainModule:
x = hexDataStr source x = hexDataStr source
check %x == %source check %x == %source
test "Odd length": test "Odd length":
expect ValueError: expect JsonRpcError:
let let
source = "0x123" source = "0x123"
x = hexDataStr source x = hexDataStr source
check %x != %source check %x != %source
test "Missing header": test "Missing header":
expect ValueError: expect JsonRpcError:
let let
source = "1234" source = "1234"
x = hexDataStr source x = hexDataStr source
check %x != %source check %x != %source
expect ValueError: expect JsonRpcError:
let let
source = "01234" source = "01234"
x = hexDataStr source x = hexDataStr source
check %x != %source check %x != %source
expect ValueError: expect JsonRpcError:
let let
source = "x1234" source = "x1234"
x = hexDataStr source x = hexDataStr source

View File

@ -1,6 +1,9 @@
import import
nimcrypto, stint, nimcrypto, stint,
ethtypes, ethhexstrings, stintjson, ../json_rpc/rpcserver ./ethtypes,
./ethhexstrings,
./stintjson,
../../json_rpc/rpcserver
#[ #[
For details on available RPC calls, see: https://github.com/ethereum/wiki/wiki/JSON-RPC For details on available RPC calls, see: https://github.com/ethereum/wiki/wiki/JSON-RPC
@ -28,6 +31,22 @@ import
specified once without invoking `reset`. specified once without invoking `reset`.
]# ]#
EthSend.useDefaultSerializationIn JrpcConv
EthCall.useDefaultSerializationIn JrpcConv
TransactionObject.useDefaultSerializationIn JrpcConv
ReceiptObject.useDefaultSerializationIn JrpcConv
FilterOptions.useDefaultSerializationIn JrpcConv
FilterData.useDefaultSerializationIn JrpcConv
LogObject.useDefaultSerializationIn JrpcConv
WhisperPost.useDefaultSerializationIn JrpcConv
WhisperMessage.useDefaultSerializationIn JrpcConv
template derefType(): untyped =
var x: BlockObject
typeof(x[])
useDefaultSerializationIn(derefType(), JrpcConv)
proc addEthRpcs*(server: RpcServer) = proc addEthRpcs*(server: RpcServer) =
server.rpc("web3_clientVersion") do() -> string: server.rpc("web3_clientVersion") do() -> string:
## Returns the current client version. ## Returns the current client version.
@ -51,10 +70,10 @@ proc addEthRpcs*(server: RpcServer) =
## "3": Ropsten Testnet ## "3": Ropsten Testnet
## "4": Rinkeby Testnet ## "4": Rinkeby Testnet
## "42": Kovan Testnet ## "42": Kovan Testnet
#[ Note, See: ## Note, See:
https://github.com/ethereum/interfaces/issues/6 ## https://github.com/ethereum/interfaces/issues/6
https://github.com/ethereum/EIPs/issues/611 ## https://github.com/ethereum/EIPs/issues/611
]#
result = "" result = ""
server.rpc("net_listening") do() -> bool: server.rpc("net_listening") do() -> bool:
@ -449,4 +468,3 @@ proc addEthRpcs*(server: RpcServer) =
## id: the filter id. ## id: the filter id.
## Returns a list of messages received since last poll. ## Returns a list of messages received since last poll.
discard discard

View File

@ -1,5 +1,5 @@
# json-rpc # json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH # Copyright (c) 2024 Status Research & Development GmbH
# Licensed under either of # Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE)) # * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT)) # * MIT license ([LICENSE-MIT](LICENSE-MIT))
@ -7,6 +7,4 @@
# This file may not be copied, modified, or distributed except according to # This file may not be copied, modified, or distributed except according to
# those terms. # those terms.
import server proc shh_uninstallFilter(id: int): bool
import servers/[socketserver, shttpserver]
export server, socketserver, shttpserver

19
tests/private/helpers.nim Normal file
View File

@ -0,0 +1,19 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
../../json_rpc/router
converter toStr*(value: distinct (string|StringOfJson)): string = string(value)
template `==`*(a: StringOfJson, b: JsonNode): bool =
parseJson(string a) == b
template `==`*(a: JsonNode, b: StringOfJson): bool =
a == parseJson(string b)

View File

@ -1,4 +1,9 @@
import stint, ../json_rpc/jsonmarshal import
std/json,
stint,
../../json_rpc/private/jrpc_conv
{.push gcsafe, raises: [].}
template stintStr(n: UInt256|Int256): JsonNode = template stintStr(n: UInt256|Int256): JsonNode =
var s = n.toHex var s = n.toHex
@ -10,13 +15,16 @@ proc `%`*(n: UInt256): JsonNode = n.stintStr
proc `%`*(n: Int256): JsonNode = n.stintStr proc `%`*(n: Int256): JsonNode = n.stintStr
proc writeValue*(w: var JsonWriter[JsonRpc], val: UInt256) = proc writeValue*(w: var JsonWriter[JrpcConv], val: UInt256)
{.gcsafe, raises: [IOError].} =
writeValue(w, val.stintStr) writeValue(w, val.stintStr)
proc writeValue*(w: var JsonWriter[JsonRpc], val: ref UInt256) = proc writeValue*(w: var JsonWriter[JrpcConv], val: ref UInt256)
{.gcsafe, raises: [IOError].} =
writeValue(w, val[].stintStr) writeValue(w, val[].stintStr)
proc readValue*(r: var JsonReader[JsonRpc], v: var UInt256) = proc readValue*(r: var JsonReader[JrpcConv], v: var UInt256)
{.gcsafe, raises: [JsonReaderError].} =
## Allows UInt256 to be passed as a json string. ## Allows UInt256 to be passed as a json string.
## Expects base 16 string, starting with "0x". ## Expects base 16 string, starting with "0x".
try: try:
@ -27,8 +35,10 @@ proc readValue*(r: var JsonReader[JsonRpc], v: var UInt256) =
except Exception as err: except Exception as err:
r.raiseUnexpectedValue("Error deserializing for '" & $v.type & "' stream: " & err.msg) r.raiseUnexpectedValue("Error deserializing for '" & $v.type & "' stream: " & err.msg)
proc readValue*(r: var JsonReader[JsonRpc], v: var ref UInt256) = proc readValue*(r: var JsonReader[JrpcConv], v: var ref UInt256)
{.gcsafe, raises: [JsonReaderError].} =
## Allows ref UInt256 to be passed as a json string. ## Allows ref UInt256 to be passed as a json string.
## Expects base 16 string, starting with "0x". ## Expects base 16 string, starting with "0x".
readValue(r, v[]) readValue(r, v[])
{.pop.}

26
tests/test_callsigs.nim Normal file
View File

@ -0,0 +1,26 @@
# json-rpc
# Copyright (c) 2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
../json_rpc/client
from os import getCurrentDir, DirSep
from strutils import rsplit
template sourceDir: string = currentSourcePath.rsplit(DirSep, 1)[0]
createRpcSigs(RpcClient, sourceDir & "/private/file_callsigs.nim")
createSingleRpcSig(RpcClient, "bottle"):
proc get_Bottle(id: int): bool
createRpcSigsFromNim(RpcClient):
proc get_Banana(id: int): bool
proc get_Combo(id, index: int, name: string): bool
proc get_Name(id: int): string
proc getJsonString(name: string): JsonString

246
tests/test_jrpc_sys.nim Normal file
View File

@ -0,0 +1,246 @@
# json-rpc
# Copyright (c) 2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
unittest2,
../json_rpc/private/jrpc_sys
func id(): RequestId =
RequestId(kind: riNull)
func id(x: string): RequestId =
RequestId(kind: riString, str: x)
func id(x: int): RequestId =
RequestId(kind: riNumber, num: x)
func req(id: int or string, meth: string, params: RequestParamsTx): RequestTx =
RequestTx(
id: Opt.some(id(id)),
`method`: meth,
params: params
)
func reqNull(meth: string, params: RequestParamsTx): RequestTx =
RequestTx(
id: Opt.some(id()),
`method`: meth,
params: params
)
func reqNoId(meth: string, params: RequestParamsTx): RequestTx =
RequestTx(
`method`: meth,
params: params
)
func toParams(params: varargs[(string, JsonString)]): seq[ParamDescNamed] =
for x in params:
result.add ParamDescNamed(name:x[0], value:x[1])
func namedPar(params: varargs[(string, JsonString)]): RequestParamsTx =
RequestParamsTx(
kind: rpNamed,
named: toParams(params)
)
func posPar(params: varargs[JsonString]): RequestParamsTx =
RequestParamsTx(
kind: rpPositional,
positional: @params
)
func res(id: int or string, r: JsonString): ResponseTx =
ResponseTx(
id: id(id),
kind: rkResult,
result: r,
)
func res(id: int or string, err: ResponseError): ResponseTx =
ResponseTx(
id: id(id),
kind: rkError,
error: err,
)
func resErr(code: int, msg: string): ResponseError =
ResponseError(
code: code,
message: msg,
)
func resErr(code: int, msg: string, data: JsonString): ResponseError =
ResponseError(
code: code,
message: msg,
data: Opt.some(data)
)
func reqBatch(args: varargs[RequestTx]): RequestBatchTx =
if args.len == 1:
RequestBatchTx(
kind: rbkSingle, single: args[0]
)
else:
RequestBatchTx(
kind: rbkMany, many: @args
)
func resBatch(args: varargs[ResponseTx]): ResponseBatchTx =
if args.len == 1:
ResponseBatchTx(
kind: rbkSingle, single: args[0]
)
else:
ResponseBatchTx(
kind: rbkMany, many: @args
)
suite "jrpc_sys conversion":
let np1 = namedPar(("banana", JsonString("true")), ("apple", JsonString("123")))
let pp1 = posPar(JsonString("123"), JsonString("true"), JsonString("\"hello\""))
test "RequestTx -> RequestRx: id(int), positional":
let tx = req(123, "int_positional", pp1)
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, RequestRx)
check:
rx.jsonrpc.isSome
rx.id.kind == riNumber
rx.id.num == 123
rx.meth.get == "int_positional"
rx.params.kind == rpPositional
rx.params.positional.len == 3
rx.params.positional[0].kind == JsonValueKind.Number
rx.params.positional[1].kind == JsonValueKind.Bool
rx.params.positional[2].kind == JsonValueKind.String
test "RequestTx -> RequestRx: id(string), named":
let tx = req("word", "string_named", np1)
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, RequestRx)
check:
rx.jsonrpc.isSome
rx.id.kind == riString
rx.id.str == "word"
rx.meth.get == "string_named"
rx.params.kind == rpNamed
rx.params.named[0].name == "banana"
rx.params.named[0].value.string == "true"
rx.params.named[1].name == "apple"
rx.params.named[1].value.string == "123"
test "RequestTx -> RequestRx: id(null), named":
let tx = reqNull("null_named", np1)
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, RequestRx)
check:
rx.jsonrpc.isSome
rx.id.kind == riNull
rx.meth.get == "null_named"
rx.params.kind == rpNamed
rx.params.named[0].name == "banana"
rx.params.named[0].value.string == "true"
rx.params.named[1].name == "apple"
rx.params.named[1].value.string == "123"
test "RequestTx -> RequestRx: none, none":
let tx = reqNoId("none_positional", posPar())
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, RequestRx)
check:
rx.jsonrpc.isSome
rx.id.kind == riNull
rx.meth.get == "none_positional"
rx.params.kind == rpPositional
rx.params.positional.len == 0
test "ResponseTx -> ResponseRx: id(int), res":
let tx = res(777, JsonString("true"))
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, ResponseRx)
check:
rx.jsonrpc.isSome
rx.id.isSome
rx.id.get.num == 777
rx.result.isSome
rx.result.get == JsonString("true")
rx.error.isNone
test "ResponseTx -> ResponseRx: id(string), err: nodata":
let tx = res("gum", resErr(999, "fatal"))
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, ResponseRx)
check:
rx.jsonrpc.isSome
rx.id.isSome
rx.id.get.str == "gum"
rx.result.isNone
rx.error.isSome
rx.error.get.code == 999
rx.error.get.message == "fatal"
rx.error.get.data.isNone
test "ResponseTx -> ResponseRx: id(string), err: some data":
let tx = res("gum", resErr(999, "fatal", JsonString("888.999")))
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, ResponseRx)
check:
rx.jsonrpc.isSome
rx.id.isSome
rx.id.get.str == "gum"
rx.result.isNone
rx.error.isSome
rx.error.get.code == 999
rx.error.get.message == "fatal"
rx.error.get.data.get == JsonString("888.999")
test "RequestBatchTx -> RequestBatchRx: single":
let tx1 = req(123, "int_positional", pp1)
let tx = reqBatch(tx1)
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, RequestBatchRx)
check:
rx.kind == rbkSingle
test "RequestBatchTx -> RequestBatchRx: many":
let tx1 = req(123, "int_positional", pp1)
let tx2 = req("word", "string_named", np1)
let tx3 = reqNull("null_named", np1)
let tx4 = reqNoId("none_positional", posPar())
let tx = reqBatch(tx1, tx2, tx3, tx4)
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, RequestBatchRx)
check:
rx.kind == rbkMany
rx.many.len == 4
test "ResponseBatchTx -> ResponseBatchRx: single":
let tx1 = res(777, JsonString("true"))
let tx = resBatch(tx1)
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, ResponseBatchRx)
check:
rx.kind == rbkSingle
test "ResponseBatchTx -> ResponseBatchRx: many":
let tx1 = res(777, JsonString("true"))
let tx2 = res("gum", resErr(999, "fatal"))
let tx3 = res("gum", resErr(999, "fatal", JsonString("888.999")))
let tx = resBatch(tx1, tx2, tx3)
let txBytes = JrpcSys.encode(tx)
let rx = JrpcSys.decode(txBytes, ResponseBatchRx)
check:
rx.kind == rbkMany
rx.many.len == 3

97
tests/test_router_rpc.nim Normal file
View File

@ -0,0 +1,97 @@
import
unittest2,
../json_rpc/router,
json_serialization/stew/results,
json_serialization/std/options
var server = RpcRouter()
server.rpc("optional") do(A: int, B: Option[int], C: string, D: Option[int], E: Option[string]) -> string:
var res = "A: " & $A
res.add ", B: " & $B.get(99)
res.add ", C: " & C
res.add ", D: " & $D.get(77)
res.add ", E: " & E.get("none")
return res
server.rpc("noParams") do() -> int:
return 123
server.rpc("emptyParams"):
return %777
server.rpc("comboParams") do(a, b, c: int) -> int:
return a+b+c
server.rpc("returnJsonString") do(a, b, c: int) -> JsonString:
return JsonString($(a+b+c))
func req(meth: string, params: string): string =
"""{"jsonrpc":"2.0", "id":0, "method": """ &
"\"" & meth & "\", \"params\": " & params & "}"
suite "rpc router":
test "no params":
let n = req("noParams", "[]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":123}"""
test "no params with params":
let n = req("noParams", "[123]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"error":{"code":-32000,"message":"noParams raised an exception","data":"Expected 0 Json parameter(s) but got 1"}}"""
test "optional B E, positional":
let n = req("optional", "[44, null, \"apple\", 33]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":"A: 44, B: 99, C: apple, D: 33, E: none"}"""
test "optional B D E, positional":
let n = req("optional", "[44, null, \"apple\"]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":"A: 44, B: 99, C: apple, D: 77, E: none"}"""
test "optional D E, positional":
let n = req("optional", "[44, 567, \"apple\"]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":"A: 44, B: 567, C: apple, D: 77, E: none"}"""
test "optional D wrong E, positional":
let n = req("optional", "[44, 567, \"apple\", \"banana\"]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"error":{"code":-32000,"message":"optional raised an exception","data":"Parameter [D] of type 'Option[system.int]' could not be decoded: number expected"}}"""
test "optional D extra, positional":
let n = req("optional", "[44, 567, \"apple\", 999, \"banana\", true]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":"A: 44, B: 567, C: apple, D: 999, E: banana"}"""
test "optional B D E, named":
let n = req("optional", """{"A": 33, "C":"banana" }""")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":"A: 33, B: 99, C: banana, D: 77, E: none"}"""
test "optional B E, D front, named":
let n = req("optional", """{"D": 8887, "A": 33, "C":"banana" }""")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":"A: 33, B: 99, C: banana, D: 8887, E: none"}"""
test "optional B E, D front, extra X, named":
let n = req("optional", """{"D": 8887, "X": false , "A": 33, "C":"banana"}""")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":"A: 33, B: 99, C: banana, D: 8887, E: none"}"""
test "empty params":
let n = req("emptyParams", "[]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":777}"""
test "combo params":
let n = req("comboParams", "[6,7,8]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":21}"""
test "return json string":
let n = req("returnJsonString", "[6,7,8]")
let res = waitFor server.route(n)
check res == """{"jsonrpc":"2.0","id":0,"result":21}"""

View File

@ -1,7 +1,20 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import import
unittest2, tables, unittest2, tables,
stint, ethtypes, ethprocs, stintjson, chronicles, stint, chronicles,
../json_rpc/[rpcclient, rpcserver], ./helpers ../json_rpc/[rpcclient, rpcserver],
./private/helpers,
./private/ethtypes,
./private/ethprocs,
./private/stintjson
from os import getCurrentDir, DirSep from os import getCurrentDir, DirSep
from strutils import rsplit from strutils import rsplit
@ -15,7 +28,7 @@ var
server.addEthRpcs() server.addEthRpcs()
## Generate client convenience marshalling wrappers from forward declarations ## Generate client convenience marshalling wrappers from forward declarations
createRpcSigs(RpcSocketClient, sourceDir & DirSep & "ethcallsigs.nim") createRpcSigs(RpcSocketClient, sourceDir & "/private/ethcallsigs.nim")
func rpcDynamicName(name: string): string = func rpcDynamicName(name: string): string =
"rpc." & name "rpc." & name
@ -38,7 +51,7 @@ proc testLocalCalls: Future[seq[StringOfJson]] =
returnUint256 = server.executeMethod("rpc.testReturnUint256", %[]) returnUint256 = server.executeMethod("rpc.testReturnUint256", %[])
return all(uint256Param, returnUint256) return all(uint256Param, returnUint256)
proc testRemoteUInt256: Future[seq[Response]] = proc testRemoteUInt256: Future[seq[StringOfJson]] =
## Call function remotely on server, testing `stint` types ## Call function remotely on server, testing `stint` types
let let
uint256Param = client.call("rpc.uint256Param", %[%"0x1234567890"]) uint256Param = client.call("rpc.uint256Param", %[%"0x1234567890"])

View File

@ -1,10 +1,19 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import import
unittest2, unittest2,
websock/websock, websock/websock,
../json_rpc/[rpcclient, rpcserver] ../json_rpc/[rpcclient, rpcserver]
const const
serverHost = "localhost" serverHost = "127.0.0.1"
serverPort = 8547 serverPort = 8547
serverAddress = serverHost & ":" & $serverPort serverAddress = serverHost & ":" & $serverPort
@ -31,18 +40,19 @@ suite "HTTP server hook test":
waitFor client.connect(serverHost, Port(serverPort), false) waitFor client.connect(serverHost, Port(serverPort), false)
expect ErrorResponse: expect ErrorResponse:
let r = waitFor client.call("testHook", %[%"abc"]) let r = waitFor client.call("testHook", %[%"abc"])
discard r
test "good auth token": test "good auth token":
let client = newRpcHttpClient(getHeaders = authHeaders) let client = newRpcHttpClient(getHeaders = authHeaders)
waitFor client.connect(serverHost, Port(serverPort), false) waitFor client.connect(serverHost, Port(serverPort), false)
let r = waitFor client.call("testHook", %[%"abc"]) let r = waitFor client.call("testHook", %[%"abc"])
check r.getStr == "Hello abc" check r.string == "\"Hello abc\""
waitFor srv.closeWait() waitFor srv.closeWait()
proc wsAuthHeaders(ctx: Hook, proc wsAuthHeaders(ctx: Hook,
headers: var HttpTable): Result[void, string] headers: var HttpTable): Result[void, string]
{.gcsafe, raises: [Defect].} = {.gcsafe, raises: [].} =
headers.add("Auth-Token", "Good Token") headers.add("Auth-Token", "Good Token")
return ok() return ok()
@ -80,7 +90,7 @@ suite "Websocket server hook test":
test "good auth token": test "good auth token":
waitFor client.connect("ws://127.0.0.1:8545/", hooks = @[hook]) waitFor client.connect("ws://127.0.0.1:8545/", hooks = @[hook])
let r = waitFor client.call("testHook", %[%"abc"]) let r = waitFor client.call("testHook", %[%"abc"])
check r.getStr == "Hello abc" check r.string == "\"Hello abc\""
srv.stop() srv.stop()
waitFor srv.closeWait() waitFor srv.closeWait()

View File

@ -1,3 +1,12 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import unittest2 import unittest2
import ../json_rpc/[rpcserver, rpcclient] import ../json_rpc/[rpcserver, rpcclient]
@ -7,7 +16,7 @@ proc simpleTest(address: string, port: Port): Future[bool] {.async.} =
var client = newRpcHttpClient() var client = newRpcHttpClient()
await client.connect(address, port, secure = false) await client.connect(address, port, secure = false)
var r = await client.call("noParamsProc", %[]) var r = await client.call("noParamsProc", %[])
if r.getStr == "Hello world": if r.string == "\"Hello world\"":
result = true result = true
proc continuousTest(address: string, port: Port): Future[int] {.async.} = proc continuousTest(address: string, port: Port): Future[int] {.async.} =
@ -16,7 +25,7 @@ proc continuousTest(address: string, port: Port): Future[int] {.async.} =
for i in 0..<TestsCount: for i in 0..<TestsCount:
await client.connect(address, port, secure = false) await client.connect(address, port, secure = false)
var r = await client.call("myProc", %[%"abc", %[1, 2, 3, i]]) var r = await client.call("myProc", %[%"abc", %[1, 2, 3, i]])
if r.getStr == "Hello abc data: [1, 2, 3, " & $i & "]": if r.string == "\"Hello abc data: [1, 2, 3, " & $i & "]\"":
result += 1 result += 1
await client.close() await client.close()
@ -27,17 +36,17 @@ proc invalidTest(address: string, port: Port): Future[bool] {.async.} =
try: try:
var r = await client.call("invalidProcA", %[]) var r = await client.call("invalidProcA", %[])
discard r discard r
except ValueError: except JsonRpcError:
invalidA = true invalidA = true
try: try:
var r = await client.call("invalidProcB", %[1, 2, 3]) var r = await client.call("invalidProcB", %[1, 2, 3])
discard r discard r
except ValueError: except JsonRpcError:
invalidB = true invalidB = true
if invalidA and invalidB: if invalidA and invalidB:
result = true result = true
var httpsrv = newRpcHttpServer(["localhost:8545"]) var httpsrv = newRpcHttpServer(["127.0.0.1:8545"])
# Create RPC on server # Create RPC on server
httpsrv.rpc("myProc") do(input: string, data: array[0..3, int]): httpsrv.rpc("myProc") do(input: string, data: array[0..3, int]):
@ -49,11 +58,11 @@ httpsrv.start()
suite "JSON-RPC test suite": suite "JSON-RPC test suite":
test "Simple RPC call": test "Simple RPC call":
check waitFor(simpleTest("localhost", Port(8545))) == true check waitFor(simpleTest("127.0.0.1", Port(8545))) == true
test "Continuous RPC calls (" & $TestsCount & " messages)": test "Continuous RPC calls (" & $TestsCount & " messages)":
check waitFor(continuousTest("localhost", Port(8545))) == TestsCount check waitFor(continuousTest("127.0.0.1", Port(8545))) == TestsCount
test "Invalid RPC calls": test "Invalid RPC calls":
check waitFor(invalidTest("localhost", Port(8545))) == true check waitFor(invalidTest("127.0.0.1", Port(8545))) == true
waitFor httpsrv.stop() waitFor httpsrv.stop()
waitFor httpsrv.closeWait() waitFor httpsrv.closeWait()

View File

@ -1,6 +1,14 @@
import unittest2, strutils # json-rpc
import httputils # Copyright (c) 2019-2023 Status Research & Development GmbH
import ../json_rpc/[rpcsecureserver, rpcclient] # Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import unittest2
import ../json_rpc/[rpcserver, rpcclient]
import chronos/[streams/tlsstream, apps/http/httpcommon] import chronos/[streams/tlsstream, apps/http/httpcommon]
const TestsCount = 100 const TestsCount = 100
@ -69,7 +77,7 @@ proc simpleTest(address: string, port: Port): Future[bool] {.async.} =
var client = newRpcHttpClient(secure=true) var client = newRpcHttpClient(secure=true)
await client.connect(address, port, secure=true) await client.connect(address, port, secure=true)
var r = await client.call("noParamsProc", %[]) var r = await client.call("noParamsProc", %[])
if r.getStr == "Hello world": if r.string == "\"Hello world\"":
result = true result = true
proc continuousTest(address: string, port: Port): Future[int] {.async.} = proc continuousTest(address: string, port: Port): Future[int] {.async.} =
@ -78,7 +86,7 @@ proc continuousTest(address: string, port: Port): Future[int] {.async.} =
for i in 0..<TestsCount: for i in 0..<TestsCount:
await client.connect(address, port, secure=true) await client.connect(address, port, secure=true)
var r = await client.call("myProc", %[%"abc", %[1, 2, 3, i]]) var r = await client.call("myProc", %[%"abc", %[1, 2, 3, i]])
if r.getStr == "Hello abc data: [1, 2, 3, " & $i & "]": if r.string == "\"Hello abc data: [1, 2, 3, " & $i & "]\"":
result += 1 result += 1
await client.close() await client.close()
@ -89,19 +97,21 @@ proc invalidTest(address: string, port: Port): Future[bool] {.async.} =
try: try:
var r = await client.call("invalidProcA", %[]) var r = await client.call("invalidProcA", %[])
discard r discard r
except ValueError: except JsonRpcError:
invalidA = true invalidA = true
try: try:
var r = await client.call("invalidProcB", %[1, 2, 3]) var r = await client.call("invalidProcB", %[1, 2, 3])
discard r discard r
except ValueError: except JsonRpcError:
invalidB = true invalidB = true
if invalidA and invalidB: if invalidA and invalidB:
result = true result = true
let secureKey = TLSPrivateKey.init(HttpsSelfSignedRsaKey) let secureKey = TLSPrivateKey.init(HttpsSelfSignedRsaKey)
let secureCert = TLSCertificate.init(HttpsSelfSignedRsaCert) let secureCert = TLSCertificate.init(HttpsSelfSignedRsaCert)
var secureHttpSrv = newRpcSecureHttpServer(["localhost:8545"], secureKey, secureCert) var secureHttpSrv = RpcHttpServer.new()
secureHttpSrv.addSecureHttpServer("127.0.0.1:8545", secureKey, secureCert)
# Create RPC on server # Create RPC on server
secureHttpSrv.rpc("myProc") do(input: string, data: array[0..3, int]): secureHttpSrv.rpc("myProc") do(input: string, data: array[0..3, int]):
@ -113,11 +123,11 @@ secureHttpSrv.start()
suite "JSON-RPC test suite": suite "JSON-RPC test suite":
test "Simple RPC call": test "Simple RPC call":
check waitFor(simpleTest("localhost", Port(8545))) == true check waitFor(simpleTest("127.0.0.1", Port(8545))) == true
test "Continuous RPC calls (" & $TestsCount & " messages)": test "Continuous RPC calls (" & $TestsCount & " messages)":
check waitFor(continuousTest("localhost", Port(8545))) == TestsCount check waitFor(continuousTest("127.0.0.1", Port(8545))) == TestsCount
test "Invalid RPC calls": test "Invalid RPC calls":
check waitFor(invalidTest("localhost", Port(8545))) == true check waitFor(invalidTest("127.0.0.1", Port(8545))) == true
waitFor secureHttpSrv.stop() waitFor secureHttpSrv.stop()
waitFor secureHttpSrv.closeWait() waitFor secureHttpSrv.closeWait()

View File

@ -1,9 +1,18 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import import
unittest2, chronicles, unittest2, chronicles,
../json_rpc/[rpcclient, rpcserver, rpcproxy] ../json_rpc/[rpcclient, rpcserver, rpcproxy]
let srvAddress = initTAddress("127.0.0.1", Port(8545)) let srvAddress = initTAddress("127.0.0.1", Port(8545))
let proxySrvAddress = "localhost:8546" let proxySrvAddress = "127.0.0.1:8546"
let proxySrvAddressForClient = "http://"&proxySrvAddress let proxySrvAddressForClient = "http://"&proxySrvAddress
template registerMethods(srv: RpcServer, proxy: RpcProxy) = template registerMethods(srv: RpcServer, proxy: RpcProxy) =
@ -29,10 +38,10 @@ suite "Proxy RPC through http":
test "Successful RPC call thorugh proxy": test "Successful RPC call thorugh proxy":
let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]]) let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]])
check r.getStr == "Hello abc data: [1, 2, 3, 4]" check r.string == "\"Hello abc data: [1, 2, 3, 4]\""
test "Successful RPC call no proxy": test "Successful RPC call no proxy":
let r = waitFor client.call("myProc1", %[%"abc", %[1, 2, 3, 4]]) let r = waitFor client.call("myProc1", %[%"abc", %[1, 2, 3, 4]])
check r.getStr == "Hello abc data: [1, 2, 3, 4]" check r.string == "\"Hello abc data: [1, 2, 3, 4]\""
test "Missing params": test "Missing params":
expect(CatchableError): expect(CatchableError):
discard waitFor client.call("myProc", %[%"abc"]) discard waitFor client.call("myProc", %[%"abc"])
@ -58,10 +67,10 @@ suite "Proxy RPC through websockets":
test "Successful RPC call thorugh proxy": test "Successful RPC call thorugh proxy":
let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]]) let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]])
check r.getStr == "Hello abc data: [1, 2, 3, 4]" check r.string == "\"Hello abc data: [1, 2, 3, 4]\""
test "Successful RPC call no proxy": test "Successful RPC call no proxy":
let r = waitFor client.call("myProc1", %[%"abc", %[1, 2, 3, 4]]) let r = waitFor client.call("myProc1", %[%"abc", %[1, 2, 3, 4]])
check r.getStr == "Hello abc data: [1, 2, 3, 4]" check r.string == "\"Hello abc data: [1, 2, 3, 4]\""
test "Missing params": test "Missing params":
expect(CatchableError): expect(CatchableError):
discard waitFor client.call("myProc", %[%"abc"]) discard waitFor client.call("myProc", %[%"abc"])

View File

@ -1,5 +1,18 @@
import unittest2, chronicles, options # json-rpc
import ../json_rpc/rpcserver, ./helpers # Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import
unittest2,
chronicles,
../json_rpc/rpcserver,
./private/helpers,
json_serialization/std/options
type type
# some nested types to check object parsing # some nested types to check object parsing
@ -26,6 +39,19 @@ type
Enum0 Enum0
Enum1 Enum1
MyObject.useDefaultSerializationIn JrpcConv
Test.useDefaultSerializationIn JrpcConv
Test2.useDefaultSerializationIn JrpcConv
MyOptional.useDefaultSerializationIn JrpcConv
MyOptionalNotBuiltin.useDefaultSerializationIn JrpcConv
proc readValue*(r: var JsonReader[JrpcConv], val: var MyEnum)
{.gcsafe, raises: [IOError, SerializationError].} =
let intVal = r.parseInt(int)
if intVal < low(MyEnum).int or intVal > high(MyEnum).int:
r.raiseUnexpectedValue("invalid enum range " & $intVal)
val = MyEnum(intVal)
let let
testObj = %*{ testObj = %*{
"a": %1, "a": %1,
@ -38,7 +64,7 @@ let
}, },
"c": %1.0} "c": %1.0}
var s = newRpcSocketServer(["localhost:8545"]) var s = newRpcSocketServer(["127.0.0.1:8545"])
# RPC definitions # RPC definitions
s.rpc("rpc.simplePath"): s.rpc("rpc.simplePath"):
@ -100,6 +126,8 @@ type
d: Option[int] d: Option[int]
e: Option[string] e: Option[string]
OptionalFields.useDefaultSerializationIn JrpcConv
s.rpc("rpc.mixedOptionalArg") do(a: int, b: Option[int], c: string, s.rpc("rpc.mixedOptionalArg") do(a: int, b: Option[int], c: string,
d: Option[int], e: Option[string]) -> OptionalFields: d: Option[int], e: Option[string]) -> OptionalFields:
@ -125,6 +153,8 @@ type
o2: Option[bool] o2: Option[bool]
o3: Option[bool] o3: Option[bool]
MaybeOptions.useDefaultSerializationIn JrpcConv
s.rpc("rpc.optInObj") do(data: string, options: Option[MaybeOptions]) -> int: s.rpc("rpc.optInObj") do(data: string, options: Option[MaybeOptions]) -> int:
if options.isSome: if options.isSome:
let o = options.get let o = options.get
@ -155,10 +185,10 @@ suite "Server types":
test "Enum param paths": test "Enum param paths":
block: block:
let r = waitFor s.executeMethod("rpc.enumParam", %[(int64(Enum1))]) let r = waitFor s.executeMethod("rpc.enumParam", %[%int64(Enum1)])
check r == "[\"Enum1\"]" check r == "[\"Enum1\"]"
expect(ValueError): expect(JsonRpcError):
discard waitFor s.executeMethod("rpc.enumParam", %[(int64(42))]) discard waitFor s.executeMethod("rpc.enumParam", %[(int64(42))])
test "Different param types": test "Different param types":
@ -201,30 +231,30 @@ suite "Server types":
inp2 = MyOptional() inp2 = MyOptional()
r1 = waitFor s.executeMethod("rpc.optional", %[%inp1]) r1 = waitFor s.executeMethod("rpc.optional", %[%inp1])
r2 = waitFor s.executeMethod("rpc.optional", %[%inp2]) r2 = waitFor s.executeMethod("rpc.optional", %[%inp2])
check r1 == JsonRpc.encode inp1 check r1.string == JrpcConv.encode inp1
check r2 == JsonRpc.encode inp2 check r2.string == JrpcConv.encode inp2
test "Return statement": test "Return statement":
let r = waitFor s.executeMethod("rpc.testReturns", %[]) let r = waitFor s.executeMethod("rpc.testReturns", %[])
check r == JsonRpc.encode 1234 check r == JrpcConv.encode 1234
test "Runtime errors": test "Runtime errors":
expect ValueError: expect JsonRpcError:
# root param not array # root param not array
discard waitFor s.executeMethod("rpc.arrayParam", %"test") discard waitFor s.executeMethod("rpc.arrayParam", %"test")
expect ValueError: expect JsonRpcError:
# too big for array # too big for array
discard waitFor s.executeMethod("rpc.arrayParam", %[%[0, 1, 2, 3, 4, 5, 6], %"hello"]) discard waitFor s.executeMethod("rpc.arrayParam", %[%[0, 1, 2, 3, 4, 5, 6], %"hello"])
expect ValueError: expect JsonRpcError:
# wrong sub parameter type # wrong sub parameter type
discard waitFor s.executeMethod("rpc.arrayParam", %[%"test", %"hello"]) discard waitFor s.executeMethod("rpc.arrayParam", %[%"test", %"hello"])
expect ValueError: expect JsonRpcError:
# wrong param type # wrong param type
discard waitFor s.executeMethod("rpc.differentParams", %[%"abc", %1]) discard waitFor s.executeMethod("rpc.differentParams", %[%"abc", %1])
test "Multiple variables of one type": test "Multiple variables of one type":
let r = waitFor s.executeMethod("rpc.multiVarsOfOneType", %[%"hello", %"world"]) let r = waitFor s.executeMethod("rpc.multiVarsOfOneType", %[%"hello", %"world"])
check r == JsonRpc.encode "hello world" check r == JrpcConv.encode "hello world"
test "Optional arg": test "Optional arg":
let let
@ -233,37 +263,37 @@ suite "Server types":
r1 = waitFor s.executeMethod("rpc.optionalArg", %[%117, %int1]) r1 = waitFor s.executeMethod("rpc.optionalArg", %[%117, %int1])
r2 = waitFor s.executeMethod("rpc.optionalArg", %[%117]) r2 = waitFor s.executeMethod("rpc.optionalArg", %[%117])
r3 = waitFor s.executeMethod("rpc.optionalArg", %[%117, newJNull()]) r3 = waitFor s.executeMethod("rpc.optionalArg", %[%117, newJNull()])
check r1 == JsonRpc.encode int1 check r1 == JrpcConv.encode int1
check r2 == JsonRpc.encode int2 check r2 == JrpcConv.encode int2
check r3 == JsonRpc.encode int2 check r3 == JrpcConv.encode int2
test "Optional arg2": test "Optional arg2":
let r1 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B"]) let r1 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B"])
check r1 == JsonRpc.encode "AB" check r1 == JrpcConv.encode "AB"
let r2 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", newJNull()]) let r2 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", newJNull()])
check r2 == JsonRpc.encode "AB" check r2 == JrpcConv.encode "AB"
let r3 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", newJNull(), newJNull()]) let r3 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", newJNull(), newJNull()])
check r3 == JsonRpc.encode "AB" check r3 == JrpcConv.encode "AB"
let r4 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", newJNull(), %"D"]) let r4 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", newJNull(), %"D"])
check r4 == JsonRpc.encode "ABD" check r4 == JrpcConv.encode "ABD"
let r5 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", %"C", %"D"]) let r5 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", %"C", %"D"])
check r5 == JsonRpc.encode "ABCD" check r5 == JrpcConv.encode "ABCD"
let r6 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", %"C", newJNull()]) let r6 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", %"C", newJNull()])
check r6 == JsonRpc.encode "ABC" check r6 == JrpcConv.encode "ABC"
let r7 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", %"C"]) let r7 = waitFor s.executeMethod("rpc.optionalArg2", %[%"A", %"B", %"C"])
check r7 == JsonRpc.encode "ABC" check r7 == JrpcConv.encode "ABC"
test "Mixed optional arg": test "Mixed optional arg":
var ax = waitFor s.executeMethod("rpc.mixedOptionalArg", %[%10, %11, %"hello", %12, %"world"]) var ax = waitFor s.executeMethod("rpc.mixedOptionalArg", %[%10, %11, %"hello", %12, %"world"])
check ax == JsonRpc.encode OptionalFields(a: 10, b: some(11), c: "hello", d: some(12), e: some("world")) check ax == JrpcConv.encode OptionalFields(a: 10, b: some(11), c: "hello", d: some(12), e: some("world"))
var bx = waitFor s.executeMethod("rpc.mixedOptionalArg", %[%10, newJNull(), %"hello"]) var bx = waitFor s.executeMethod("rpc.mixedOptionalArg", %[%10, newJNull(), %"hello"])
check bx == JsonRpc.encode OptionalFields(a: 10, c: "hello") check bx == JrpcConv.encode OptionalFields(a: 10, c: "hello")
test "Non-built-in optional types": test "Non-built-in optional types":
let let
@ -271,33 +301,33 @@ suite "Server types":
testOpts1 = MyOptionalNotBuiltin(val: some(t2)) testOpts1 = MyOptionalNotBuiltin(val: some(t2))
testOpts2 = MyOptionalNotBuiltin() testOpts2 = MyOptionalNotBuiltin()
var r = waitFor s.executeMethod("rpc.optionalArgNotBuiltin", %[%testOpts1]) var r = waitFor s.executeMethod("rpc.optionalArgNotBuiltin", %[%testOpts1])
check r == JsonRpc.encode t2.y check r == JrpcConv.encode t2.y
var r2 = waitFor s.executeMethod("rpc.optionalArgNotBuiltin", %[]) var r2 = waitFor s.executeMethod("rpc.optionalArgNotBuiltin", %[])
check r2 == JsonRpc.encode "Empty1" check r2 == JrpcConv.encode "Empty1"
var r3 = waitFor s.executeMethod("rpc.optionalArgNotBuiltin", %[%testOpts2]) var r3 = waitFor s.executeMethod("rpc.optionalArgNotBuiltin", %[%testOpts2])
check r3 == JsonRpc.encode "Empty2" check r3 == JrpcConv.encode "Empty2"
test "Manually set up JSON for optionals": test "Manually set up JSON for optionals":
# Check manual set up json with optionals # Check manual set up json with optionals
let opts1 = parseJson("""{"o1": true}""") let opts1 = parseJson("""{"o1": true}""")
var r1 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts1]) var r1 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts1])
check r1 == JsonRpc.encode 1 check r1 == JrpcConv.encode 1
let opts2 = parseJson("""{"o2": true}""") let opts2 = parseJson("""{"o2": true}""")
var r2 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts2]) var r2 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts2])
check r2 == JsonRpc.encode 2 check r2 == JrpcConv.encode 2
let opts3 = parseJson("""{"o3": true}""") let opts3 = parseJson("""{"o3": true}""")
var r3 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts3]) var r3 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts3])
check r3 == JsonRpc.encode 4 check r3 == JrpcConv.encode 4
# Combinations # Combinations
let opts4 = parseJson("""{"o1": true, "o3": true}""") let opts4 = parseJson("""{"o1": true, "o3": true}""")
var r4 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts4]) var r4 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts4])
check r4 == JsonRpc.encode 5 check r4 == JrpcConv.encode 5
let opts5 = parseJson("""{"o2": true, "o3": true}""") let opts5 = parseJson("""{"o2": true, "o3": true}""")
var r5 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts5]) var r5 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts5])
check r5 == JsonRpc.encode 6 check r5 == JrpcConv.encode 6
let opts6 = parseJson("""{"o1": true, "o2": true}""") let opts6 = parseJson("""{"o1": true, "o2": true}""")
var r6 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts6]) var r6 = waitFor s.executeMethod("rpc.optInObj", %[%"0x31ded", opts6])
check r6 == JsonRpc.encode 3 check r6 == JrpcConv.encode 3
s.stop() s.stop()
waitFor s.closeWait() waitFor s.closeWait()

View File

@ -1,5 +1,14 @@
# json-rpc
# Copyright (c) 2019-2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE))
# * MIT license ([LICENSE-MIT](LICENSE-MIT))
# at your option.
# This file may not be copied, modified, or distributed except according to
# those terms.
import import
unittest2, unittest2,
../json_rpc/[rpcclient, rpcserver] ../json_rpc/[rpcclient, rpcserver]
# Create RPC on server # Create RPC on server
@ -14,16 +23,16 @@ proc setupServer*(srv: RpcServer) =
raise (ref InvalidRequest)(code: -32001, msg: "Unknown payload") raise (ref InvalidRequest)(code: -32001, msg: "Unknown payload")
suite "Socket Server/Client RPC": suite "Socket Server/Client RPC":
var srv = newRpcSocketServer(["localhost:8545"]) var srv = newRpcSocketServer(["127.0.0.1:8545"])
var client = newRpcSocketClient() var client = newRpcSocketClient()
srv.setupServer() srv.setupServer()
srv.start() srv.start()
waitFor client.connect("localhost", Port(8545)) waitFor client.connect("127.0.0.1", Port(8545))
test "Successful RPC call": test "Successful RPC call":
let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]]) let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]])
check r.getStr == "Hello abc data: [1, 2, 3, 4]" check r.string == "\"Hello abc data: [1, 2, 3, 4]\""
test "Missing params": test "Missing params":
expect(CatchableError): expect(CatchableError):
@ -38,7 +47,7 @@ suite "Socket Server/Client RPC":
discard waitFor client.call("invalidRequest", %[]) discard waitFor client.call("invalidRequest", %[])
check false check false
except CatchableError as e: except CatchableError as e:
check e.msg == """{"code":-32001,"message":"Unknown payload","data":null}""" check e.msg == """{"code":-32001,"message":"Unknown payload"}"""
srv.stop() srv.stop()
waitFor srv.closeWait() waitFor srv.closeWait()
@ -53,7 +62,7 @@ suite "Websocket Server/Client RPC":
test "Successful RPC call": test "Successful RPC call":
let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]]) let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]])
check r.getStr == "Hello abc data: [1, 2, 3, 4]" check r.string == "\"Hello abc data: [1, 2, 3, 4]\""
test "Missing params": test "Missing params":
expect(CatchableError): expect(CatchableError):
@ -68,7 +77,7 @@ suite "Websocket Server/Client RPC":
discard waitFor client.call("invalidRequest", %[]) discard waitFor client.call("invalidRequest", %[])
check false check false
except CatchableError as e: except CatchableError as e:
check e.msg == """{"code":-32001,"message":"Unknown payload","data":null}""" check e.msg == """{"code":-32001,"message":"Unknown payload"}"""
srv.stop() srv.stop()
waitFor srv.closeWait() waitFor srv.closeWait()
@ -85,7 +94,7 @@ suite "Websocket Server/Client RPC with Compression":
test "Successful RPC call": test "Successful RPC call":
let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]]) let r = waitFor client.call("myProc", %[%"abc", %[1, 2, 3, 4]])
check r.getStr == "Hello abc data: [1, 2, 3, 4]" check r.string == "\"Hello abc data: [1, 2, 3, 4]\""
test "Missing params": test "Missing params":
expect(CatchableError): expect(CatchableError):
@ -100,7 +109,8 @@ suite "Websocket Server/Client RPC with Compression":
discard waitFor client.call("invalidRequest", %[]) discard waitFor client.call("invalidRequest", %[])
check false check false
except CatchableError as e: except CatchableError as e:
check e.msg == """{"code":-32001,"message":"Unknown payload","data":null}""" check e.msg == """{"code":-32001,"message":"Unknown payload"}"""
srv.stop() srv.stop()
waitFor srv.closeWait() waitFor srv.closeWait()