status-im
/
airbyte-custom-connector
mirror of https://github.com/status-im/airbyte-custom-connector.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

crypto-market-extractor: Failsafe API 

Signed-off-by: Alexis Pentori <alexis@status.im>
This commit is contained in:
Alexis Pentori 2024-02-08 11:06:03 +01:00
parent 691a633300
commit 067652009e
No known key found for this signature in database
GPG Key ID: 65250D2801E47A10
4 changed files with 17 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
source-crypto-market-extractor/README.md
View File

@ -47,9 +47,9 @@ pip install -r requirements.txt
### Locally running the connector ### Locally running the connector
``` ```
python main.py spec python main.py spec
python main.py check --config sample_files/coin_list.json python main.py check --config sample_files/config-example.json
python main.py discover --config sample_files/coin_list.json python main.py discover --config sample_files/config-example.json
python main.py read --config sample_files/coin_list.json --catalog sample_files/configured_catalog.json python main.py read --config sample_files/config-example.json --catalog sample_files/configured_catalog.json
``` ```
### Locally running the connector docker image ### Locally running the connector docker image

2
source-crypto-market-extractor/metadata.yaml
View File

@ -10,7 +10,7 @@ data:
connectorSubtype: api connectorSubtype: api
connectorType: source connectorType: source
definitionId: ce91e290-be98-4a34-b43b-28330afdc3c9 definitionId: ce91e290-be98-4a34-b43b-28330afdc3c9
dockerImageTag: 0.0.3 dockerImageTag: 0.1.0
dockerRepository: harbor.status.im/status-im/airbyte/crypto-market-extractor dockerRepository: harbor.status.im/status-im/airbyte/crypto-market-extractor
githubIssueLabel: source-crypto-market-extractor githubIssueLabel: source-crypto-market-extractor
icon: crypto-market-extractor.svg icon: crypto-market-extractor.svg

0
source-crypto-market-extractor/sample_files/coin_list.json → source-crypto-market-extractor/sample_files/config-example.json
View File

20
source-crypto-market-extractor/source_crypto_market_extractor/source.py
View File

@ -44,13 +44,19 @@ class CoinPrice(HttpStream):
coin=stream_slice["coin"] coin=stream_slice["coin"]
logger.info("Parsing Coin Gecko data for %s", coin) logger.info("Parsing Coin Gecko data for %s", coin)
market_chart = response.json() market_chart = response.json()
yield { logger.info("Response: %s", market_chart)
"name": coin, data={ "name": coin, "date": datetime.today().strftime('%Y%m%d_%H%M')}
"date": datetime.today().strftime('%Y%m%d_%H%M'), try:
# The first value of the prices objects is weird if len(market_chart) > 1:
"price": market_chart['prices'][1][1], data['price'] = market_chart['prices'][1][1]
} elif len(market_chart) == 1:
data['price'] = market_chart['prices'][0][1]
else:
logger.error("Invalid response from API, %s", market_chart)
raise "No correct data return"
except err:
logger.error('An error happened : %s', err)
yield data
# Source # Source
class SourceCryptoMarketExtractor(AbstractSource): class SourceCryptoMarketExtractor(AbstractSource):
def check_connection(self, logger, config) -> Tuple[bool, any]: def check_connection(self, logger, config) -> Tuple[bool, any]: