status-im/consul
2
0
Fork 0
mirror of https://github.com/status-im/consul.git synced 2025-01-10 22:06:20 +00:00
Code Issues Projects Releases Wiki Activity
consul/.circleci/config.yml
John Cowen b3b32dc0f6
ui: UI Release Merge (ui-staging merge) (#6527) 
## HTTPAdapter (#5637)

## Ember upgrade 2.18 > 3.12 (#6448)

### Proxies can no longer get away with not calling _super

This means that we can't use create anymore to define dynamic methods.
Therefore we dynamically make 2 extended Proxies on demand, and then
create from those. Therefore we can call _super in the init method of
the extended Proxies.

### We aren't allowed to reset a service anymore

We never actually need to now anyway, this is a remnant of the refactor
from browser based confirmations. We fix it as simply as possible here
but will revisit and remove the old browser confirm functionality at a
later date

### Revert classes to use ES5 style to workaround babel transp. probs

Using a mixture of ES6 classes (and hence super) and arrow functions
means that when babel transpiles the arrow functions down to ES5, a
reference to this is moved before the call to super, hence causing a js
error.

Furthermore, we the testing environment no longer lets use use
apply/call on the constructor.

These errors only manifests during testing (only in the testing
environment), the application itself runs fine with no problems without
this change.

Using ES5 style class definitions give us freedom to do all of the above
without causing any errors, so we reverted these classes back to ES5
class definitions

### Skip test that seems to have changed due to a change in RSVP timing

This test tests a usecase/area of the API that will probably never ever
be used, it was more testing out the API. We've skipped the test for now
as this doesn't affect the application itself, but left a note to come
back here later to investigate further

### Remove enumerableContentDidChange

Initial testing looks like we don't need to call this function anymore,
the function no longer exists

### Rework Changeset.isSaving to take into account new ember APIs

Setting/hanging a computedProperty of an instantiated object no longer
works. Move to setting it on the prototype/class definition instead

### Change how we detect whether something requires listening

New ember API's have changed how you can detect whether something is a
computedProperty or not. It's not immediately clear if its even possible
now. Therefore we change how we detect whether something should be
listened to or not by just looking for presence of `addEventListener`

### Potentially temporary change of ci test scripts to ensure deps exist

All our tooling scripts run through a Makefile (for people familiar with
only using those), which then call yarn scripts which can be called
independently (for people familar with only using yarn).

The Makefile targets always check to make sure all the dependencies are
installed before running anything that requires them (building, testing
etc).

The CI scripts/targets didn't follow this same route and called the yarn
scripts directly (usually CI builds a cache of the dependencies first).

For some reason this cache isn't doing what it usually does, and it
looks as though, in CI, ember isn't installed.

This commit makes the CI scripts consistently use the same method as all
of the other tooling scripts (Makefile target > Install Deps if
required > call yarn script). This should install the dependencies if
for some reason the CI cache building doesn't complete/isn't successful.

Potentially this commit may be reverted if, the root of the problem is
elsewhere, although consistency is always good, so it might be a good
idea to leave this commit as is even if we need to debug and fix things
elsewhere.

### Make test-parallel consistent with the rest of the tooling scripts

As we are here making changes for CI purposes (making test-ci
consistent), we spotted that test-parallel is also inconsistent and also
the README manual instructions won't work without `ember` installed
globally.

This commit makes everything consistent and changes the manual
instructions to use the local ember instance that gets installed via
yarn

### Re-wrangle catchable to fit with new ember 3.12 APIs

In the upgrade from ember 3.8 > 3.12 the public interfaces for
ComputedProperties have changed slightly. `meta` is no longer a public
property of ComputedProperty but of a ComputedDecoratorImpl mixin
instead.

7e4ba1096e/packages/%40ember/-internals/metal/lib/computed.ts (L725)

There seems to be no way, by just using publically available
methods, to replicate this behaviour so that we can create our own
'ComputedProperty` factory via injecting the ComputedProperty class as
we did previously.

3f333bada1/ui-v2/app/utils/computed/factory.js (L1-L18)

Instead we dynamically hang our `Catchable` `catch` method off the
instantiated ComputedProperty. In doing it like this `ComputedProperty`
has already has its `meta` method mixed in so we don't have to manually
mix it in ourselves (which doesn't seem possible)

This functionality is only used during our work in trying to ensure
our EventSource/BlockingQuery work was as 'ember-like' as possible (i.e.
using the traditional Route.model hooks and ember-like Controller
properties). Our ongoing/upcoming work on a componentized approach to
data a.k.a `<DataSource />` means we will be able to remove the majority
of the code involved here now that it seems to be under an amount of
flux in ember.

### Build bindata_assetfs.go with new UI changes
2019-09-30 14:47:49 +01:00

604 lines
17 KiB
YAML
Raw Blame History

---
version: 2
references:
images:
go: &GOLANG_IMAGE circleci/golang:1.12.8
middleman: &MIDDLEMAN_IMAGE hashicorp/middleman-hashicorp:0.3.35
ember: &EMBER_IMAGE circleci/node:8-browsers
paths:
test-results: &TEST_RESULTS_DIR /tmp/test-results
cache:
yarn: &YARN_CACHE_KEY consul-ui-v1-{{ checksum "ui-v2/yarn.lock" }}
rubygem: &RUBYGEM_CACHE_KEY static-site-gems-v1-{{ checksum "Gemfile.lock" }}
environment: &ENVIRONMENT
TEST_RESULTS_DIR: *TEST_RESULTS_DIR
GOTESTSUM_RELEASE: 0.3.3
EMAIL: noreply@hashicorp.com
GIT_AUTHOR_NAME: circleci-consul
GIT_COMMITTER_NAME: circleci-consul
S3_ARTIFACT_BUCKET: consul-dev-artifacts
BASH_ENV: .circleci/bash_env.sh
VAULT_BINARY_VERSION: 1.2.2
jobs:
# lint consul tests
lint-consul-retry:
docker:
- image: *GOLANG_IMAGE
steps:
- checkout
- run: go get -u github.com/hashicorp/lint-consul-retry && lint-consul-retry
# Runs go fmt and go vet
go-fmt-and-vet:
docker:
- image: *GOLANG_IMAGE
steps:
- checkout
- restore_cache:
keys:
- consul-modcache-v1-{{ checksum "go.mod" }}
- run:
command: go mod download
- save_cache:
key: consul-modcache-v1-{{ checksum "go.mod" }}
paths:
- /go/pkg/mod
- run:
name: check go fmt
command: |
files=$(go fmt ./...)
if [ -n "$files" ]; then
echo "The following file(s) do not conform to go fmt:"
echo "$files"
exit 1
fi
- run:
command: go vet ./...
environment:
<<: *ENVIRONMENT
# checks vendor directory is correct
check-vendor:
docker:
- image: *GOLANG_IMAGE
environment:
<<: *ENVIRONMENT
steps:
- checkout
- restore_cache:
keys:
- consul-modcache-v1-{{ checksum "go.mod" }}
- run:
command: make update-vendor
- save_cache:
key: consul-modcache-v1-{{ checksum "go.mod" }}
paths:
- /go/pkg/mod
- run: |
if ! git diff --exit-code; then
echo "Git directory has vendor changes"
exit 1
fi
go-test:
docker:
- image: *GOLANG_IMAGE
parallelism: 8
environment:
<<: *ENVIRONMENT
GOTAGS: "" # No tags for OSS but there are for enterprise
steps:
- checkout
- restore_cache: # restore cache from dev-build job
keys:
- consul-modcache-v1-{{ checksum "go.mod" }}
- attach_workspace:
at: /go/bin
- run: mkdir -p $TEST_RESULTS_DIR
- run: sudo apt-get update && sudo apt-get install -y rsyslog
- run: sudo service rsyslog start
- run: |
wget -q -O /tmp/vault.zip https://releases.hashicorp.com/vault/${VAULT_BINARY_VERSION}/vault_${VAULT_BINARY_VERSION}_linux_amd64.zip
sudo unzip -d /usr/local/bin /tmp/vault.zip
rm -rf /tmp/vault*
- run: |
PACKAGE_NAMES=$(go list ./... | circleci tests split --split-by=timings --timings-type=classname)
gotestsum --format=short-verbose --junitfile $TEST_RESULTS_DIR/gotestsum-report.xml -- -tags=$GOTAGS -p 3 $PACKAGE_NAMES
- store_test_results:
path: /tmp/test-results
- store_artifacts:
path: /tmp/test-results
# split off a job for the API package since it is separate
go-test-api:
docker:
- image: *GOLANG_IMAGE
environment:
<<: *ENVIRONMENT
GOTAGS: "" # No tags for OSS but there are for enterprise
steps:
- checkout
- restore_cache: # restore cache from dev-build job
keys:
- consul-modcache-v1-{{ checksum "go.mod" }}
- attach_workspace:
at: /go/bin
- run: mkdir -p $TEST_RESULTS_DIR
- run:
working_directory: api
command: |
PACKAGE_NAMES=$(go list ./... | circleci tests split --split-by=timings --timings-type=classname)
gotestsum --format=short-verbose --junitfile $TEST_RESULTS_DIR/gotestsum-report.xml -- -tags=$GOTAGS $PACKAGE_NAMES
- store_test_results:
path: /tmp/test-results
- store_artifacts:
path: /tmp/test-results
# build all distros
build-distros: &build-distros
docker:
- image: *GOLANG_IMAGE
environment: &build-env
<<: *ENVIRONMENT
GOXPARALLEL: 2 # CircleCI containers are 2 CPU x 4GB RAM
resource_class: large
steps:
- checkout
- run: |
make tools
# restore any go.{mod,sum} files that were updated indirectly
git reset --hard HEAD
- run: ./build-support/scripts/build-local.sh
# save dev build to CircleCI
- store_artifacts:
path: ./pkg/bin
# build all 386 architecture supported OS binaries
build-386:
<<: *build-distros
environment:
<<: *build-env
XC_OS: "darwin freebsd linux windows"
XC_ARCH: "386"
# build all amd64 architecture supported OS binaries
build-amd64:
<<: *build-distros
environment:
<<: *build-env
XC_OS: "darwin freebsd linux solaris windows"
XC_ARCH: "amd64"
# build all arm/arm64 architecture supported OS binaries
build-arm:
docker:
- image: *GOLANG_IMAGE
environment:
<<: *ENVIRONMENT
CGO_ENABLED: 1
GOOS: linux
steps:
- checkout
- run: sudo apt-get update && sudo apt-get install -y gcc-arm-linux-gnueabi gcc-arm-linux-gnueabihf gcc-aarch64-linux-gnu
- run:
environment:
GOARM: 5
CC: arm-linux-gnueabi-gcc
GOARCH: arm
command: go build -o ./pkg/bin/linux_armel/consul -ldflags="${GOLDFLAGS}"
- run:
environment:
GOARM: 6
CC: arm-linux-gnueabihf-gcc
GOARCH: arm
command: go build -o ./pkg/bin/linux_armhf/consul -ldflags="${GOLDFLAGS}"
- run:
environment:
CC: aarch64-linux-gnu-gcc
GOARCH: arm64
command: go build -o ./pkg/bin/linux_aarch64/consul -ldflags="${GOLDFLAGS}"
- store_artifacts:
path: ./pkg/bin
# create a development build
dev-build:
docker:
- image: *GOLANG_IMAGE
environment:
<<: *ENVIRONMENT
steps:
- checkout
- restore_cache:
keys:
- consul-modcache-v1-{{ checksum "go.mod" }}
- run:
command: make dev
- save_cache:
key: consul-modcache-v1-{{ checksum "go.mod" }}
paths:
- /go/pkg/mod
# save dev build to pass to downstream jobs
- persist_to_workspace:
root: /go/bin
paths:
- consul
# upload development build to s3
dev-upload-s3:
docker:
- image: circleci/python:stretch
environment:
<<: *ENVIRONMENT
steps:
- run:
name: Install awscli
command: sudo pip install awscli
# get consul binary
- attach_workspace:
at: bin/
- run:
name: package binary
command: tar -czf consul.tar.gz -C bin/ .
- run:
name: Upload to s3
command: |
if [ -n "${S3_ARTIFACT_PATH}" ]; then
aws s3 cp \
--metadata "CIRCLECI=${CIRCLECI},CIRCLE_BUILD_URL=${CIRCLE_BUILD_URL},CIRCLE_BRANCH=${CIRCLE_BRANCH}" \
"consul.tar.gz" "s3://${S3_ARTIFACT_BUCKET}/${S3_ARTIFACT_PATH}/${CIRCLE_SHA1}.tar.gz"
else
echo "CircleCI - S3_ARTIFACT_PATH was not set"
exit 1
fi
# upload dev docker image
dev-upload-docker:
docker:
- image: circleci/golang:latest # use a circleci image so the attach_workspace step works (has ca-certs installed)
environment:
<<: *ENVIRONMENT
steps:
- checkout
# get consul binary
- attach_workspace:
at: bin/
- setup_remote_docker:
docker_layer_caching: true
- run: make ci.dev-docker
# Nomad 0.8 builds on go0.10
# Run integration tests on nomad/v0.8.7
nomad-integration-0_8:
docker:
- image: circleci/golang:1.10
environment:
<<: *ENVIRONMENT
NOMAD_WORKING_DIR: &NOMAD_WORKING_DIR /go/src/github.com/hashicorp/nomad
NOMAD_VERSION: v0.8.7
steps: &NOMAD_INTEGRATION_TEST_STEPS
- run: git clone https://github.com/hashicorp/nomad.git --branch ${NOMAD_VERSION} ${NOMAD_WORKING_DIR}
# get consul binary
- attach_workspace:
at: /go/bin
# make test result directory
- run: mkdir -p $TEST_RESULTS_DIR
# make dev build of nomad
- run:
command: make dev
working_directory: *NOMAD_WORKING_DIR
# update gotestsum
- run: curl -sSL "https://github.com/gotestyourself/gotestsum/releases/download/v${GOTESTSUM_RELEASE}/gotestsum_${GOTESTSUM_RELEASE}_linux_amd64.tar.gz" | sudo tar --overwrite -xz -C /usr/local/bin gotestsum
# run integration tests
- run:
command: gotestsum --format=short-verbose --junitfile $TEST_RESULTS_DIR/results.xml -- ./command/agent/consul -run TestConsul
working_directory: *NOMAD_WORKING_DIR
# store test results for CircleCI
- store_test_results:
path: *TEST_RESULTS_DIR
- store_artifacts:
path: *TEST_RESULTS_DIR
# run integration tests on nomad/master
nomad-integration-master:
docker:
- image: *GOLANG_IMAGE
environment:
<<: *ENVIRONMENT
NOMAD_WORKING_DIR: /go/src/github.com/hashicorp/nomad
NOMAD_VERSION: master
steps: *NOMAD_INTEGRATION_TEST_STEPS
build-website:
# setting the working_directory along with the checkout path allows us to not have
# to cd into the website/ directory for commands
working_directory: ~/project/website
docker:
- image: *MIDDLEMAN_IMAGE
steps:
- checkout:
path: ~/project
# restores gem cache
- restore_cache:
key: *RUBYGEM_CACHE_KEY
- run:
name: install gems
command: bundle check || bundle install --path vendor/bundle --retry=3
# saves gem cache if we have changed the Gemfile
- save_cache:
key: *RUBYGEM_CACHE_KEY
paths:
- ~/project/website/vendor/bundle
# exclude guides directory since they moved to learn.hashicorp.com
# keep index.html which points to learn
- run:
name: exclude guides
command: find ./source/docs/guides -type f -not -name 'index.html.md' -delete
- run:
name: middleman build
command: bundle exec middleman build
- run:
name: add missing tar binary
command: apk update && apk add tar
# saves website build directory
- persist_to_workspace:
root: .
paths:
- build
deploy-website:
# setting the working_directory along with the checkout path allows us to not have
# to cd into the website/ directory for commands
working_directory: ~/project/website
docker:
- image: *MIDDLEMAN_IMAGE
steps:
- checkout:
path: ~/project
- run:
name: add missing tar binary
command: apk update && apk add tar
# attach website build directory
- attach_workspace:
at: ~/project/website
# restores gem cache
- restore_cache:
key: *RUBYGEM_CACHE_KEY
# rerun build with 'ENV=production' to add analytics
- run:
name: install gems
command: bundle check || bundle install --path vendor/bundle --retry=3
# exclude guides directory since they moved to learn.hashicorp.com
# keep index.html which points to learn
- run:
name: exclude guides
command: find ./source/docs/guides -type f -not -name 'index.html.md' -delete
# rerun build with 'ENV=production' to add analytics
- run:
name: middleman build
command: bundle exec middleman build
- run:
name: website deploy
command: ./scripts/deploy.sh
# Link check on a temporary netlify deployed site
docs-link-checker:
docker:
- image: circleci/node:lts
steps:
- checkout
# attach website build directory
- attach_workspace:
at: ~/project/website
- run: ./website/scripts/link-check.sh
# build frontend yarn cache
frontend-cache:
docker:
- image: *EMBER_IMAGE
steps:
- checkout
# cache yarn deps
- restore_cache:
key: *YARN_CACHE_KEY
- run:
name: install yarn packages
command: cd ui-v2 && yarn install
- save_cache:
key: *YARN_CACHE_KEY
paths:
- ui-v2/node_modules
# build ember so frontend tests run faster
ember-build:
docker:
- image: *EMBER_IMAGE
steps:
- checkout
- restore_cache:
key: *YARN_CACHE_KEY
- run: cd ui-v2 && make build-ci
# saves the build to a workspace to be passed to a downstream job
- persist_to_workspace:
root: ui-v2
paths:
- dist
# run ember frontend tests
ember-test:
docker:
- image: *EMBER_IMAGE
environment:
EMBER_TEST_PARALLEL: true #enables test parallelization with ember-exam
EMBER_TEST_REPORT: test-results/report.xml #outputs test report for CircleCI test summary
parallelism: 4
steps:
- checkout
- restore_cache:
key: *YARN_CACHE_KEY
- attach_workspace:
at: ui-v2
- run:
working_directory: ui-v2
command: node_modules/ember-cli/bin/ember exam --split=$CIRCLE_NODE_TOTAL --partition=`expr $CIRCLE_NODE_INDEX + 1` --path dist --silent -r xunit
- store_test_results:
path: ui-v2/test-results
# Envoy integration tests. Require docker dev binary to be built already
envoy-integration-test-1.8.0:
docker:
# We only really need bash and docker-compose which is installed on all
# Circle images but pick Go since we have to pick one of them.
- image: *GOLANG_IMAGE
environment:
ENVOY_VERSIONS: "1.8.0"
steps: &ENVOY_INTEGRATION_TEST_STEPS
- checkout
# Get go binary from workspace
- attach_workspace:
at: .
- setup_remote_docker:
docker_layer_caching: true
# Build the consul-dev image from the already built binary
- run: docker build -t consul-dev -f ./build-support/docker/Consul-Dev.dockerfile .
- run:
name: Envoy Integration Tests
command: make test-envoy-integ SKIP_DOCKER_BUILD=1
environment:
# tput complains if this isn't set to something.
TERM: ansi
- store_artifacts:
path: ./test/integration/connect/envoy/workdir/logs
destination: container-logs
envoy-integration-test-1.9.1:
docker:
- image: *GOLANG_IMAGE
environment:
ENVOY_VERSIONS: "1.9.1"
steps: *ENVOY_INTEGRATION_TEST_STEPS
envoy-integration-test-1.10.0:
docker:
- image: *GOLANG_IMAGE
environment:
ENVOY_VERSIONS: "1.10.0"
steps: *ENVOY_INTEGRATION_TEST_STEPS
envoy-integration-test-1.11.1:
docker:
- image: *GOLANG_IMAGE
environment:
ENVOY_VERSIONS: "1.11.1"
steps: *ENVOY_INTEGRATION_TEST_STEPS
workflows:
version: 2
go-tests:
jobs:
- check-vendor
- lint-consul-retry
- go-fmt-and-vet:
requires:
- check-vendor
- dev-build:
requires:
- go-fmt-and-vet
- go-test: &go-test
requires:
- dev-build
- go-test-api: *go-test
build-distros:
jobs:
- build-386
- build-amd64
- build-arm
test-integrations:
jobs:
- dev-build
- dev-upload-s3: &dev-upload
requires:
- dev-build
filters:
branches:
ignore:
- /^pull\/.*$/ # only push dev builds from non forks
- dev-upload-docker:
<<: *dev-upload
context: consul-ci
- nomad-integration-master:
requires:
- dev-build
- nomad-integration-0_8:
requires:
- dev-build
- envoy-integration-test-1.8.0:
requires:
- dev-build
- envoy-integration-test-1.9.1:
requires:
- dev-build
- envoy-integration-test-1.10.0:
requires:
- dev-build
- envoy-integration-test-1.11.1:
requires:
- dev-build
website:
jobs:
- build-website
- docs-link-checker:
requires:
- build-website
filters:
branches:
ignore:
- /^pull\/.*$/ # only run link checker on non forks
- deploy-website:
requires:
- docs-link-checker
context: static-sites
filters:
branches:
only: stable-website
frontend:
jobs:
- frontend-cache
- ember-build:
requires:
- frontend-cache
- ember-test:
requires:
- ember-build
Reference in New Issue View Git Blame Copy Permalink