status-im
/
consul
mirror of https://github.com/status-im/consul.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
consul/GNUmakefile

235 lines
7.1 KiB
Makefile
Raw Normal View History

build: Use bash for building
2017-04-27 18:49:43 +00:00
SHELL = bash
Add `tools/cmd/cover` to GOTOOLS
2016-05-07 20:02:12 +00:00
GOTOOLS = \
github.com/elazarl/go-bindata-assetfs/... \
Switches to HashiCorp fork of jteeuwen/go-bindata. This was reported in #3868. We make a HashiCorp hard fork of the jteeuwen/go-bindata hard fork that was replaced and diffed the code against a Dec 1, 2015 copy of the original repository we had as a cross-check of that hard fork. This replaces references to jteeuwen/go-bindata to point to the HashiCorp fork. Fixes #3868
2018-02-07 15:36:39 +00:00
github.com/hashicorp/go-bindata/... \
build: use vendorfmt to make vendor.json more merge friendly
2017-07-18 04:46:25 +00:00
github.com/magiconair/vendorfmt/cmd/vendorfmt \
Add `tools/cmd/cover` to GOTOOLS
2016-05-07 20:02:12 +00:00
github.com/mitchellh/gox \
golang.org/x/tools/cmd/cover \
Missing tools in makefile (#3205)
2017-06-29 10:55:01 +00:00
golang.org/x/tools/cmd/stringer \
github.com/axw/gocov/gocov \
gopkg.in/matm/v1/gocov-html
build: build all packages together Build all packages together with a global timeout. Locally, the tests pass within 40 sec. On travis, we complete within 3-4 min. travis truncates the logs after 4MB which we are hitting so show the relevant information first and then whatever is left.
2017-06-29 12:50:47 +00:00
build: drop 'consul' build tag for OSS build
2017-08-30 11:25:14 +00:00
GOTAGS ?=
build: Simplify make test and log output
2017-04-26 22:47:57 +00:00
GOFILES ?= $(shell go list ./... | grep -v /vendor/)
Travis evaluates ENV before cloning git repo and cding so we need to delay gathering packages until the makefile
2018-02-21 12:53:35 +00:00
ifeq ($(origin GOTEST_PKGS_EXCLUDE), undefined)
Split the heavy test packages out to their own Jobs.
2018-02-21 12:20:33 +00:00
GOTEST_PKGS ?= "./..."
Travis evaluates ENV before cloning git repo and cding so we need to delay gathering packages until the makefile
2018-02-21 12:53:35 +00:00
else
GOTEST_PKGS=$(shell go list ./... | sed 's/github.com\/hashicorp\/consul/./' | egrep -v "^($(GOTEST_PKGS_EXCLUDE))$$")
endif
Fix a couple find warnings on linux Additionally add the ability to use go install for dev builds rather than gox (travis doesn’t have gox)
2018-06-19 14:49:07 +00:00
GOOS?=$(shell go env GOOS)
GOARCH?=$(shell go env GOARCH)
Allow for default GOPATH
2018-01-04 19:38:20 +00:00
GOPATH=$(shell go env GOPATH)
Faster dev builds (#2924) This patch runs 'go install' instead of gox which runs 'go build' for 'make dev' and copies the binary into ./bin and ./pkg/${GOOS}_${GOARCH} to mimick the previous behavior. This reduces the roundtrip times for a dev build from 11 sec to 500ms if there weren't any changes.
2017-04-18 22:03:51 +00:00
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
ASSETFS_PATH?=agent/bindata_assetfs.go
Faster dev builds (#2924) This patch runs 'go install' instead of gox which runs 'go build' for 'make dev' and copies the binary into ./bin and ./pkg/${GOOS}_${GOARCH} to mimick the previous behavior. This reduces the roundtrip times for a dev build from 11 sec to 500ms if there weren't any changes.
2017-04-18 22:03:51 +00:00
# Get the git commit
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
GIT_COMMIT?=$(shell git rev-parse --short HEAD)
GIT_DIRTY?=$(shell test -n "`git status --porcelain`" && echo "+CHANGES" || true)
GIT_DESCRIBE?=$(shell git describe --tags --always)
Faster dev builds (#2924) This patch runs 'go install' instead of gox which runs 'go build' for 'make dev' and copies the binary into ./bin and ./pkg/${GOOS}_${GOARCH} to mimick the previous behavior. This reduces the roundtrip times for a dev build from 11 sec to 500ms if there weren't any changes.
2017-04-18 22:03:51 +00:00
GIT_IMPORT=github.com/hashicorp/consul/version
GOLDFLAGS=-X $(GIT_IMPORT).GitCommit=$(GIT_COMMIT)$(GIT_DIRTY) -X $(GIT_IMPORT).GitDescribe=$(GIT_DESCRIBE)
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
ifeq ($(FORCE_REBUILD),1)
NOCACHE=--no-cache
else
NOCACHE=
endif
DOCKER_BUILD_QUIET?=1
ifeq (${DOCKER_BUILD_QUIET},1)
QUIET=-q
else
QUIET=
endif
CONSUL_DEV_IMAGE?=consul-dev
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
GO_BUILD_TAG?=consul-build-go
UI_BUILD_TAG?=consul-build-ui
UI_LEGACY_BUILD_TAG?=consul-build-ui-legacy
BUILD_CONTAINER_NAME?=consul-builder
Redo the build system Improvements: - More modular - Building within docker doesn’t use volumes so can be run on a remote docker host - Build containers include only minimal context so they only rarely need to be rebuilt and most of the time can be used from the cache. - 3 build containers instead of 1. One based off of the upstream golang containers for building go stuff with all our required GOTOOLS installed. One like the old container based off ubuntu bionic for building the old UI (didn’t bother creating a much better container as this shouldn’t be needed once we completely remove the legacy UI). One for building the new UI. Its alpine based with all the node, ember, yarn stuff installed. - Top level makefile has the ability to do a container based build without running make dist - Can build for arbitrary platforms at the top level using: make consul-docker XC_OS=… XC_ARCH=… - overridable functionality to allow for customizations to the enterprise build (like to generate multiple binaries) - unified how we compile our go. always use gox even for dev-builds or rather always use the tooling around our scripts which will make sure things get copied to the correct places throughout the filesystem.
2018-06-12 20:55:52 +00:00
DIST_TAG?=1
DIST_BUILD?=1
DIST_SIGN?=1
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
ifdef DIST_VERSION
Quote some make vars
2018-06-18 18:36:24 +00:00
DIST_VERSION_ARG=-v "$(DIST_VERSION)"
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
else
DIST_VERSION_ARG=
endif
ifdef DIST_RELEASE_DATE
Quote some make vars
2018-06-18 18:36:24 +00:00
DIST_DATE_ARG=-d "$(DIST_RELEASE_DATE)"
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
else
DIST_DATE_ARG=
endif
Allow for building pre-releases/rcs/betas
2018-06-18 21:01:20 +00:00
ifdef DIST_PRERELEASE
DIST_REL_ARG=-r "$(DIST_PRERELEASE)"
else
DIST_REL_ARG=
endif
Generalize git pushing in a bash function
2018-06-15 19:23:26 +00:00
PUB_GIT?=1
PUB_WEBSITE?=1
ifeq ($(PUB_GIT),1)
PUB_GIT_ARG=-g
else
PUB_GIT_ARG=
endif
ifeq ($(PUB_WEBSITE),1)
PUB_WEBSITE_ARG=-g
else
PUB_WEBSITE_ARG=
endif
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
export GO_BUILD_TAG
export UI_BUILD_TAG
export UI_LEGACY_BUILD_TAG
export BUILD_CONTAINER_NAME
export GIT_COMMIT
export GIT_DIRTY
export GIT_DESCRIBE
export GOTAGS
Faster dev builds (#2924) This patch runs 'go install' instead of gox which runs 'go build' for 'make dev' and copies the binary into ./bin and ./pkg/${GOOS}_${GOARCH} to mimick the previous behavior. This reduces the roundtrip times for a dev build from 11 sec to 500ms if there weren't any changes.
2017-04-18 22:03:51 +00:00
export GOLDFLAGS
consul: adding basic skeleton
2013-12-06 23:43:07 +00:00
Tweaks some of the default makefile targets.
2016-02-18 04:36:48 +00:00
# all builds binaries for all targets
Only builds the binary stuff under master branch CI.
2016-10-25 20:49:57 +00:00
all: bin
Add capability to put tree back into dev mode via make dev-tree
2018-06-15 12:00:12 +00:00
bin: tools dev-build
Use gox for building
2015-10-22 18:16:01 +00:00
Typos
2015-10-22 19:00:35 +00:00
# dev creates binaries for testing locally - these are put into ./bin and $GOPATH
build: add changelogfmt target to format [GH-xxxx] references to links
2017-10-04 18:56:28 +00:00
dev: changelogfmt vendorfmt dev-build
build: fix travis build
2017-07-18 07:22:49 +00:00
dev-build:
Fix a couple find warnings on linux Additionally add the ability to use go install for dev builds rather than gox (travis doesn’t have gox)
2018-06-19 14:49:07 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/build-local.sh -o $(GOOS) -a $(GOARCH)
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
dev-docker:
@docker build -t '$(CONSUL_DEV_IMAGE)' --build-arg 'GIT_COMMIT=$(GIT_COMMIT)' --build-arg 'GIT_DIRTY=$(GIT_DIRTY)' --build-arg 'GIT_DESCRIBE=$(GIT_DESCRIBE)' -f $(CURDIR)/build-support/docker/Consul-Dev.dockerfile $(CURDIR)
Use gox for building
2015-10-22 18:16:01 +00:00
build: fix travis build
2017-07-18 07:22:49 +00:00
vendorfmt:
New config parser, HCL support, multiple bind addrs (#3480) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt
2017-09-25 18:40:42 +00:00
@echo "--> Formatting vendor/vendor.json"
build: fix travis build
2017-07-18 07:22:49 +00:00
test -x $(GOPATH)/bin/vendorfmt || go get -u github.com/magiconair/vendorfmt/cmd/vendorfmt
vendorfmt
build: add changelogfmt target to format [GH-xxxx] references to links
2017-10-04 18:56:28 +00:00
changelogfmt:
@echo "--> Making [GH-xxxx] references clickable..."
@sed -E 's|([^\[])\[GH-([0-9]+)\]|\1[[GH-\2](https://github.com/hashicorp/consul/issues/\2)]|g' CHANGELOG.md > changelog.tmp && mv changelog.tmp CHANGELOG.md
fix type in make file
2017-06-28 14:48:00 +00:00
# linux builds a linux package independent of the source platform
Add make target for linux binary
2017-05-04 11:31:56 +00:00
linux:
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/build-local.sh -o linux -a amd64
Add make target for linux binary
2017-05-04 11:31:56 +00:00
Tweaks some of the default makefile targets.
2016-02-18 04:36:48 +00:00
# dist builds binaries for all platforms and packages them for distribution
dist:
Allow for building pre-releases/rcs/betas
2018-06-18 21:01:20 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/release.sh -t '$(DIST_TAG)' -b '$(DIST_BUILD)' -S '$(DIST_SIGN)' $(DIST_VERSION_ARG) $(DIST_DATE_ARG) $(DIST_REL_ARG)
Add more functionality related to verifying a build and publishing
2018-06-13 19:10:02 +00:00
publish:
Quote some make vars
2018-06-18 18:36:24 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/publish.sh $(PUB_GIT_ARG) $(PUB_WEBSITE_ARG)
Use gox for building
2015-10-22 18:16:01 +00:00
Add capability to put tree back into dev mode via make dev-tree
2018-06-15 12:00:12 +00:00
dev-tree:
@$(SHELL) $(CURDIR)/build-support/scripts/dev.sh
consul: adding basic skeleton
2013-12-06 23:43:07 +00:00
cov:
build: Simplify make test and log output
2017-04-26 22:47:57 +00:00
gocov test $(GOFILES) | gocov-html > /tmp/coverage.html
consul: adding basic skeleton
2013-12-06 23:43:07 +00:00
open /tmp/coverage.html
test: replace porter tool with freeport lib This patch removes the porter tool which hands out free ports from a given range with a library which does the same thing. The challenge for acquiring free ports in concurrent go test runs is that go packages are tested concurrently and run in separate processes. There has to be some inter-process synchronization in preventing processes allocating the same ports. freeport allocates blocks of ports from a range expected to be not in heavy use and implements a system-wide mutex by binding to the first port of that block for the lifetime of the application. Ports are then provided sequentially from that block and are tested on localhost before being returned as available.
2017-10-21 19:15:01 +00:00
test: other-consul dev-build vet
New config parser, HCL support, multiple bind addrs (#3480) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt
2017-09-25 18:40:42 +00:00
@echo "--> Running go test"
@rm -f test.log exit-code
Split the heavy test packages out to their own Jobs.
2018-02-21 12:20:33 +00:00
go test -tags '$(GOTAGS)' -i $(GOTEST_PKGS)
Only output sparse lines to keep Travis happy while logging verbosely to disk
2018-02-20 15:32:44 +00:00
@# Dump verbose output to test.log so we can surface test names on failure but
@# hide it from travis as it exceeds their log limits and causes job to be
@# terminated (over 4MB and over 10k lines in the UI). We need to output
@# _something_ to stop them terminating us due to inactivity...
Split the heavy test packages out to their own Jobs.
2018-02-21 12:20:33 +00:00
{ go test $(GOTEST_FLAGS) -tags '$(GOTAGS)' -timeout 5m $(GOTEST_PKGS) 2>&1 ; echo $$? > exit-code ; } | tee test.log | egrep '^(ok|FAIL)\s*github.com/hashicorp/consul'
New config parser, HCL support, multiple bind addrs (#3480) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt
2017-09-25 18:40:42 +00:00
@echo "Exit code: $$(cat exit-code)" >> test.log
build: build all packages together Build all packages together with a global timeout. Locally, the tests pass within 40 sec. On travis, we complete within 3-4 min. travis truncates the logs after 4MB which we are hitting so show the relevant information first and then whatever is left.
2017-06-29 12:50:47 +00:00
@grep -A5 'DATA RACE' test.log || true
@grep -A10 'panic: test timed out' test.log || true
New config parser, HCL support, multiple bind addrs (#3480) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt
2017-09-25 18:40:42 +00:00
@grep -A1 -- '--- SKIP:' test.log || true
build: build all packages together Build all packages together with a global timeout. Locally, the tests pass within 40 sec. On travis, we complete within 3-4 min. travis truncates the logs after 4MB which we are hitting so show the relevant information first and then whatever is left.
2017-06-29 12:50:47 +00:00
@grep -A1 -- '--- FAIL:' test.log || true
@grep '^FAIL' test.log || true
New config parser, HCL support, multiple bind addrs (#3480) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt
2017-09-25 18:40:42 +00:00
@if [ "$$(cat exit-code)" == "0" ] ; then echo "PASS" ; exit 0 ; else exit 1 ; fi
build: build all packages together Build all packages together with a global timeout. Locally, the tests pass within 40 sec. On travis, we complete within 3-4 min. travis truncates the logs after 4MB which we are hitting so show the relevant information first and then whatever is left.
2017-06-29 12:50:47 +00:00
test-race:
$(MAKE) GOTEST_FLAGS=-race
build: add target for running tests with race detector
2017-05-22 19:24:38 +00:00
New config parser, HCL support, multiple bind addrs (#3480) * new config parser for agent This patch implements a new config parser for the consul agent which makes the following changes to the previous implementation: * add HCL support * all configuration fragments in tests and for default config are expressed as HCL fragments * HCL fragments can be provided on the command line so that they can eventually replace the command line flags. * HCL/JSON fragments are parsed into a temporary Config structure which can be merged using reflection (all values are pointers). The existing merge logic of overwrite for values and append for slices has been preserved. * A single builder process generates a typed runtime configuration for the agent. The new implementation is more strict and fails in the builder process if no valid runtime configuration can be generated. Therefore, additional validations in other parts of the code should be removed. The builder also pre-computes all required network addresses so that no address/port magic should be required where the configuration is used and should therefore be removed. * Upgrade github.com/hashicorp/hcl to support int64 * improve error messages * fix directory permission test * Fix rtt test * Fix ForceLeave test * Skip performance test for now until we know what to do * Update github.com/hashicorp/memberlist to update log prefix * Make memberlist use the default logger * improve config error handling * do not fail on non-existing data-dir * experiment with non-uniform timeouts to get a handle on stalled leader elections * Run tests for packages separately to eliminate the spurious port conflicts * refactor private address detection and unify approach for ipv4 and ipv6. Fixes #2825 * do not allow unix sockets for DNS * improve bind and advertise addr error handling * go through builder using test coverage * minimal update to the docs * more coverage tests fixed * more tests * fix makefile * cleanup * fix port conflicts with external port server 'porter' * stop test server on error * do not run api test that change global ENV concurrently with the other tests * Run remaining api tests concurrently * no need for retry with the port number service * monkey patch race condition in go-sockaddr until we understand why that fails * monkey patch hcl decoder race condidtion until we understand why that fails * monkey patch spurious errors in strings.EqualFold from here * add test for hcl decoder race condition. Run with go test -parallel 128 * Increase timeout again * cleanup * don't log port allocations by default * use base command arg parsing to format help output properly * handle -dc deprecation case in Build * switch autopilot.max_trailing_logs to int * remove duplicate test case * remove unused methods * remove comments about flag/config value inconsistencies * switch got and want around since the error message was misleading. * Removes a stray debug log. * Removes a stray newline in imports. * Fixes TestACL_Version8. * Runs go fmt. * Adds a default case for unknown address types. * Reoders and reformats some imports. * Adds some comments and fixes typos. * Reorders imports. * add unix socket support for dns later * drop all deprecated flags and arguments * fix wrong field name * remove stray node-id file * drop unnecessary patch section in test * drop duplicate test * add test for LeaveOnTerm and SkipLeaveOnInt in client mode * drop "bla" and add clarifying comment for the test * split up tests to support enterprise/non-enterprise tests * drop raft multiplier and derive values during build phase * sanitize runtime config reflectively and add test * detect invalid config fields * fix tests with invalid config fields * use different values for wan sanitiziation test * drop recursor in favor of recursors * allow dns_config.udp_answer_limit to be zero * make sure tests run on machines with multiple ips * Fix failing tests in a few more places by providing a bind address in the test * Gets rid of skipped TestAgent_CheckPerformanceSettings and adds case for builder. * Add porter to server_test.go to make tests there less flaky * go fmt
2017-09-25 18:40:42 +00:00
other-consul:
@echo "--> Checking for other consul instances"
@if ps -ef | grep 'consul agent' | grep -v grep ; then \
echo "Found other running consul agents. This may affect your tests." ; \
exit 1 ; \
fi
Manage dependencies via Godep Embrace the future and use Go 1.6's vendor support via Godep. Go 1.5 users should `export GO15VENDOREXPERIMENT=1`
2016-02-13 00:50:37 +00:00
cover:
build: Simplify make test and log output
2017-04-26 22:47:57 +00:00
go test $(GOFILES) --cover
consul: adding basic skeleton
2013-12-06 23:43:07 +00:00
Manage dependencies via Godep Embrace the future and use Go 1.6's vendor support via Godep. Go 1.5 users should `export GO15VENDOREXPERIMENT=1`
2016-02-13 00:50:37 +00:00
format:
Add format task to all and silence large output
2014-05-06 05:45:54 +00:00
@echo "--> Running go fmt"
build: Simplify make test and log output
2017-04-26 22:47:57 +00:00
@go fmt $(GOFILES)
Add make format task
2014-05-06 05:38:21 +00:00
Makefile: add vet target Add a vet target in order to catch suspicious constructs reported by go vet. Vet has successfully detected problems in the past, for example, see c9333b1b9b472feb5cad80e2c8276d41b64bde88 Some vet flags are noisy. In particular, the following flags reports a large amount of generally unharmful constructs: ``` -assign: check for useless assignments -composites: check that composite literals used field-keyed elements -shadow: check for shadowed variables -shadowstrict: whether to be strict about shadowing -unreachable: check for unreachable code ``` In order to skip running the flags mentioned above, vet is invoked on a directory basis with `go tool vet .` since package- level type-checking with `go vet` doesn't accept flags. Hence, each file is vetted in isolation, which is weaker than package-level type-checking. But nevertheless, it might catch suspicious constructs that pose a real issue. The vet target runs the following flags on the entire repo: ``` -asmdecl: check assembly against Go declarations -atomic: check for common mistaken usages of the sync/atomic package -bool: check for mistakes involving boolean operators -buildtags: check that +build tags are valid -copylocks: check that locks are not passed by value -methods: check that canonically named methods are canonically defined -nilfunc: check for comparisons between functions and nil -printf: check printf-like invocations -rangeloops: check that range loop variables are used correctly -shift: check for useless shifts -structtags: check that struct field tags have canonical format and apply to exported fields as needed -unsafeptr: check for misuse of unsafe.Pointer ``` Now and then, it might make sense to check the output of the disabled flags manually. For example, `VETARGS=-unreachable make vet` can detect several lines of dead code that can be deleted, etc.
2015-01-17 06:44:10 +00:00
vet:
Modernize makefile a bit
2017-03-23 23:06:25 +00:00
@echo "--> Running go vet"
Use GOTAGS in the vet make goal
2018-02-22 23:57:09 +00:00
@go vet -tags '$(GOTAGS)' $(GOFILES); if [ $$? -eq 1 ]; then \
Makefile: add vet target Add a vet target in order to catch suspicious constructs reported by go vet. Vet has successfully detected problems in the past, for example, see c9333b1b9b472feb5cad80e2c8276d41b64bde88 Some vet flags are noisy. In particular, the following flags reports a large amount of generally unharmful constructs: ``` -assign: check for useless assignments -composites: check that composite literals used field-keyed elements -shadow: check for shadowed variables -shadowstrict: whether to be strict about shadowing -unreachable: check for unreachable code ``` In order to skip running the flags mentioned above, vet is invoked on a directory basis with `go tool vet .` since package- level type-checking with `go vet` doesn't accept flags. Hence, each file is vetted in isolation, which is weaker than package-level type-checking. But nevertheless, it might catch suspicious constructs that pose a real issue. The vet target runs the following flags on the entire repo: ``` -asmdecl: check assembly against Go declarations -atomic: check for common mistaken usages of the sync/atomic package -bool: check for mistakes involving boolean operators -buildtags: check that +build tags are valid -copylocks: check that locks are not passed by value -methods: check that canonically named methods are canonically defined -nilfunc: check for comparisons between functions and nil -printf: check printf-like invocations -rangeloops: check that range loop variables are used correctly -shift: check for useless shifts -structtags: check that struct field tags have canonical format and apply to exported fields as needed -unsafeptr: check for misuse of unsafe.Pointer ``` Now and then, it might make sense to check the output of the disabled flags manually. For example, `VETARGS=-unreachable make vet` can detect several lines of dead code that can be deleted, etc.
2015-01-17 06:44:10 +00:00
echo ""; \
echo "Vet found suspicious constructs. Please check the reported constructs"; \
Modernize makefile a bit
2017-03-23 23:06:25 +00:00
echo "and fix them if necessary before submitting the code for review."; \
exit 1; \
Makefile: add vet target Add a vet target in order to catch suspicious constructs reported by go vet. Vet has successfully detected problems in the past, for example, see c9333b1b9b472feb5cad80e2c8276d41b64bde88 Some vet flags are noisy. In particular, the following flags reports a large amount of generally unharmful constructs: ``` -assign: check for useless assignments -composites: check that composite literals used field-keyed elements -shadow: check for shadowed variables -shadowstrict: whether to be strict about shadowing -unreachable: check for unreachable code ``` In order to skip running the flags mentioned above, vet is invoked on a directory basis with `go tool vet .` since package- level type-checking with `go vet` doesn't accept flags. Hence, each file is vetted in isolation, which is weaker than package-level type-checking. But nevertheless, it might catch suspicious constructs that pose a real issue. The vet target runs the following flags on the entire repo: ``` -asmdecl: check assembly against Go declarations -atomic: check for common mistaken usages of the sync/atomic package -bool: check for mistakes involving boolean operators -buildtags: check that +build tags are valid -copylocks: check that locks are not passed by value -methods: check that canonically named methods are canonically defined -nilfunc: check for comparisons between functions and nil -printf: check printf-like invocations -rangeloops: check that range loop variables are used correctly -shift: check for useless shifts -structtags: check that struct field tags have canonical format and apply to exported fields as needed -unsafeptr: check for misuse of unsafe.Pointer ``` Now and then, it might make sense to check the output of the disabled flags manually. For example, `VETARGS=-unreachable make vet` can detect several lines of dead code that can be deleted, etc.
2015-01-17 06:44:10 +00:00
fi
Fixes checked in web assets and associated build scripts. (#3173)
2017-06-21 21:43:07 +00:00
# If you've run "make ui" manually then this will get called for you. This is
# also run as part of the release build script when it verifies that there are no
# changes to the UI assets that aren't checked in.
Manage dependencies via Godep Embrace the future and use Go 1.6's vendor support via Godep. Go 1.5 users should `export GO15VENDOREXPERIMENT=1`
2016-02-13 00:50:37 +00:00
static-assets:
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
@go-bindata-assetfs -pkg agent -prefix pkg -o $(ASSETFS_PATH) ./pkg/web_ui/...
agent: compile web assets into consul binary
2015-11-30 19:24:08 +00:00
$(MAKE) format
Add capability to put tree back into dev mode via make dev-tree
2018-06-15 12:00:12 +00:00
# Build the static web ui and build static assets inside a Docker container
ui: ui-legacy-docker ui-docker static-assets-docker
Add a tools target that fetches various build-time tools
2016-02-13 01:09:18 +00:00
tools:
go get -u -v $(GOTOOLS)
Redo the build system Improvements: - More modular - Building within docker doesn’t use volumes so can be run on a remote docker host - Build containers include only minimal context so they only rarely need to be rebuilt and most of the time can be used from the cache. - 3 build containers instead of 1. One based off of the upstream golang containers for building go stuff with all our required GOTOOLS installed. One like the old container based off ubuntu bionic for building the old UI (didn’t bother creating a much better container as this shouldn’t be needed once we completely remove the legacy UI). One for building the new UI. Its alpine based with all the node, ember, yarn stuff installed. - Top level makefile has the ability to do a container based build without running make dist - Can build for arbitrary platforms at the top level using: make consul-docker XC_OS=… XC_ARCH=… - overridable functionality to allow for customizations to the enterprise build (like to generate multiple binaries) - unified how we compile our go. always use gox even for dev-builds or rather always use the tooling around our scripts which will make sure things get copied to the correct places throughout the filesystem.
2018-06-12 20:55:52 +00:00
version:
Add more functionality related to verifying a build and publishing
2018-06-13 19:10:02 +00:00
@echo -n "Version: "
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/version.sh
Add more functionality related to verifying a build and publishing
2018-06-13 19:10:02 +00:00
@echo -n "Version + release: "
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/version.sh -r
Add more functionality related to verifying a build and publishing
2018-06-13 19:10:02 +00:00
@echo -n "Version + git: "
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/version.sh -g
Add more functionality related to verifying a build and publishing
2018-06-13 19:10:02 +00:00
@echo -n "Version + release + git: "
Add capability to put tree back into dev mode via make dev-tree
2018-06-15 12:00:12 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/version.sh -r -g
Add more functionality related to verifying a build and publishing
2018-06-13 19:10:02 +00:00
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
docker-images: go-build-image ui-build-image ui-legacy-build-image
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
go-build-image:
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@echo "Building Golang build container"
@docker build $(NOCACHE) $(QUIET) --build-arg 'GOTOOLS=$(GOTOOLS)' -t $(GO_BUILD_TAG) - < build-support/docker/Build-Go.dockerfile
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
ui-build-image:
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@echo "Building UI build container"
@docker build $(NOCACHE) $(QUIET) -t $(UI_BUILD_TAG) - < build-support/docker/Build-UI.dockerfile
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
ui-legacy-build-image:
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@echo "Building Legacy UI build container"
@docker build $(NOCACHE) $(QUIET) -t $(UI_LEGACY_BUILD_TAG) - < build-support/docker/Build-UI-Legacy.dockerfile
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
static-assets-docker: go-build-image
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh static-assets
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
Redo the build system Improvements: - More modular - Building within docker doesn’t use volumes so can be run on a remote docker host - Build containers include only minimal context so they only rarely need to be rebuilt and most of the time can be used from the cache. - 3 build containers instead of 1. One based off of the upstream golang containers for building go stuff with all our required GOTOOLS installed. One like the old container based off ubuntu bionic for building the old UI (didn’t bother creating a much better container as this shouldn’t be needed once we completely remove the legacy UI). One for building the new UI. Its alpine based with all the node, ember, yarn stuff installed. - Top level makefile has the ability to do a container based build without running make dist - Can build for arbitrary platforms at the top level using: make consul-docker XC_OS=… XC_ARCH=… - overridable functionality to allow for customizations to the enterprise build (like to generate multiple binaries) - unified how we compile our go. always use gox even for dev-builds or rather always use the tooling around our scripts which will make sure things get copied to the correct places throughout the filesystem.
2018-06-12 20:55:52 +00:00
consul-docker: go-build-image
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh consul
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
ui-docker: ui-build-image
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh ui
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
ui-legacy-docker: ui-legacy-build-image
Update the scripting Automated putting the source tree into release mode.
2018-06-15 01:25:59 +00:00
@$(SHELL) $(CURDIR)/build-support/scripts/build-docker.sh ui-legacy
Initial progress on build system updates
2018-06-08 14:20:54 +00:00
.PHONY: all ci bin dev dist cov test cover format vet ui static-assets tools vendorfmt
Redo the build system Improvements: - More modular - Building within docker doesn’t use volumes so can be run on a remote docker host - Build containers include only minimal context so they only rarely need to be rebuilt and most of the time can be used from the cache. - 3 build containers instead of 1. One based off of the upstream golang containers for building go stuff with all our required GOTOOLS installed. One like the old container based off ubuntu bionic for building the old UI (didn’t bother creating a much better container as this shouldn’t be needed once we completely remove the legacy UI). One for building the new UI. Its alpine based with all the node, ember, yarn stuff installed. - Top level makefile has the ability to do a container based build without running make dist - Can build for arbitrary platforms at the top level using: make consul-docker XC_OS=… XC_ARCH=… - overridable functionality to allow for customizations to the enterprise build (like to generate multiple binaries) - unified how we compile our go. always use gox even for dev-builds or rather always use the tooling around our scripts which will make sure things get copied to the correct places throughout the filesystem.
2018-06-12 20:55:52 +00:00
.PHONY: docker-images go-build-image ui-build-image ui-legacy-build-image static-assets-docker consul-docker ui-docker ui-legacy-docker version