Commit Graph

40 Commits

Author SHA1 Message Date
R.B. Boyer 235747d473
deployer: add a bunch of test coverage and fix a few panics (#20694)
This adds a bunch of coverage of the topology.Compile method. It is not complete, but it is a start.

- A few panics and miscellany were fixed.
- The testing/deployer tests are now also run in CI.
2024-02-22 13:31:50 -06:00
Michael Zalimeni 76b5de5039
[NET-4968] Upgrade Go to 1.21 (#20062)
* Upgrade Go to 1.21

* ci: detect Go backwards compatibility test version automatically

For our submodules and other places we choose to test against previous
Go versions, detect this version automatically from the current one
rather than hard-coding it.
2024-01-12 09:57:38 -05:00
John Murret 3fa4a21edd
remove the skipping of slow tests in go-tests-ce and go-test-enterprise (#20139)
* remove the skipping of slow tests in go-tests-ce and go-test-enterprise

* add license header
2024-01-10 20:39:34 -07:00
Michael Zalimeni 3b111277ad
ci: Single source of truth for Go version in CI and Dockerfile (#20058)
ci: Set Go version consistently via .go-version

Ensure Go version is determined consistently for CI and Docker builds
rather than spread across several different files.

The intent is to eventually replace this with use of the `toolchain`
directive in Go 1.21.
2024-01-08 17:18:41 -05:00
Michael Zalimeni 1d9234a87a
ci: sanitize commit message for Slack failure alerts (#19876)
To ensure that shell code cannot be injected, capture the commit message
in an env var, then format it as needed.

Also fix several other issues with formatting and JSON escaping by
wrapping the entire message in a `toJSON` expression.
2023-12-08 16:04:45 -05:00
Michael Zalimeni 3a78446114
ci: fix escaping for Slack failure notifications (#19838)
Allow '()', '#', and other bash-interpretable special characters by
properly quoting the commit message when shortening.
2023-12-06 21:00:30 +00:00
Matt Keeler efe279f802
Retry lint fixes (#19151)
* Add a make target to run lint-consul-retry on all the modules
* Cleanup sdk/testutil/retry
* Fix a bunch of retry.Run* usage to not use the outer testing.T
* Fix some more recent retry lint issues and pin to v1.4.0 of lint-consul-retry
* Fix codegen copywrite lint issues
* Don’t perform cleanup after each retry attempt by default.
* Use the common testutil.TestingTB interface in test-integ/tenancy
* Fix retry tests
* Update otel access logging extension test to perform requests within the retry block
2023-12-06 12:11:32 -05:00
Michael Zalimeni aca8a185ca
ci: fix test failure Slack notifications (#19766)
- Skip notifications for cancelled workflows. Cancellation can be
manual or caused by branch concurrency limits.
- Fix multi-line JSON parsing error by only printing the summary line
of the commit message. We do not need more than this in Slack.
- Update Slack webhook name to match purpose.
2023-12-05 10:24:04 -05:00
Matt Keeler 8f7f15e430
Pin lint-consul-retry to v1.3.0 (#19781)
Co-authored-by: R.B. Boyer <4903+rboyer@users.noreply.github.com>
2023-11-29 22:44:22 +00:00
John Murret 2591318c82
Skip tests with p95 greater than 30 seconds outside of main and release branches. (#19628)
Skip tests with p95 greater than 30 seconds.
2023-11-15 13:43:33 -07:00
John Murret bc26fbcf2c
notify on go-tests failure on main and release branches. (#19640)
notify on failures in go-tests on main and release branches.
2023-11-15 00:42:28 +00:00
Semir Patel c7307cae6f
unhack: remove consulprem build tag (#19633) 2023-11-14 14:02:18 -06:00
Anita Akaeze f5985fedce
do not trigger integration tests (#18948) 2023-09-21 19:10:34 +00:00
Anita Akaeze 0236c48369
Update base ref property name (#18851)
* Update base ref property name

* Test skip ci (#18924)

test_push_merge

* cleanup test push code
2023-09-20 14:33:30 -07:00
Dhia Ayachi b1688ad856
Run copyright after running deep-copy as part of the Makefile/CI (#18741)
* execute copyright headers after performing deep-copy generation.

* fix copyright install

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* Apply suggestions from code review

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>

* rename steps to match codegen naming

* remove copywrite install category

---------

Co-authored-by: Semir Patel <semir.patel@hashicorp.com>
2023-09-11 13:50:52 -04:00
Curt Bushko f2ce472ae1
PLAT-1192 - Run CI on smaller instances (#18624)
Use smaller runners
2023-08-31 10:56:59 -04:00
Curt Bushko 7bc30e989c
Set concurrency for workflows (#18567)
* Set concurrency for workflows
2023-08-28 12:05:25 -04:00
Anita Akaeze 63fa78a141
NET-5382 & PLAT-1159: Do not trigger workflow if only doc files are in commit history (#18528)
NET-5382: Do not run workflow runs if at least one path matches
2023-08-23 15:26:05 -04:00
Semir Patel 53e28a4963
OSS -> CE (community edition) changes (#18517) 2023-08-22 09:46:03 -05:00
Matt Keeler b4cdfbbc12
Disable deep-copy codegen verification for now. (#18446)
Once we figure out what to do with license headers for these files we should re-enable this check.
2023-08-11 10:15:45 -04:00
Dan Bond 9b540e29bc
go-tests: disable s390x (#18273) 2023-07-25 10:58:12 -07:00
John Murret 639210e28d
Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes" (#18248)
Revert "NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236)"

This reverts commit a11dba710e.
2023-07-24 10:05:43 -06:00
John Murret a11dba710e
NET-4996 - filter go-tests and test-integration workflows from running on docs only and ui only changes (#18236) 2023-07-23 13:08:15 -06:00
hashicorp-tsccr[bot] 1ef5dfcfc6
SEC-090: Automated trusted workflow pinning (2023-07-18) (#18174)
Result of tsccr-helper -log-level=info -pin-all-workflows .

Co-authored-by: hashicorp-tsccr[bot] <hashicorp-tsccr[bot]@users.noreply.github.com>
2023-07-19 16:56:50 -07:00
John Murret 5af73901a2
[NET-4897] net/http host header is now verified and request.host that contains socked now error (#18129)
### Description

This is related to https://github.com/hashicorp/consul/pull/18124 where
we pinned the go versions in CI to 1.20.5 and 1.19.10.

go 1.20.6 and 1.19.11 now validate request host headers for validity,
including the hostname cannot be prefixed with slashes.

For local communications (npipe://, unix://), the hostname is not used,
but we need valid and meaningful hostname. Prior versions go Go would
clean the host header, and strip slashes in the process, but go1.20.6
and go1.19.11 no longer do, and reject the host header. Around the
community we are seeing that others are intercepting the req.host and if
it starts with a slash or ends with .sock, they changing the host to
localhost or another dummy value.

[client: define a "dummy" hostname to use for local connections by
thaJeztah · Pull Request #45942 ·
moby/moby](https://github.com/moby/moby/pull/45942)

### Testing & Reproduction steps

Check CI tests.

### Links
* [ ] updated test coverage
* [ ] external facing docs updated
* [ ] appropriate backport labels added
* [ ] not a security concern
2023-07-14 14:53:27 -06:00
John Murret a2c6953d0d
[NET-4895] ci - api tests and consul container tests error because of dependency bugs with go 1.20.6. Pin go to 1.20.5. (#18124)
### Description
The following jobs started failing when go 1.20.6 was released:
- `go-test-api-1-19`
- `go-test-api-1-20`
- `compatibility-integration-tests`
- `upgrade-integration-tests`

`compatibility-integration-tests` and `compatibility-integration-tests`
to this testcontainers issue:
https://github.com/testcontainers/testcontainers-go/issues/1359. This
issue calls for testcontainers to release a new version when one of
their dependencies is fixed. When that is done, we will unpin the go
versions in `compatibility-integration-tests` and
`compatibility-integration-tests`.

### Testing & Reproduction steps

See these jobs broken in CI and then see them work with this PR.

---------

Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com>
2023-07-13 19:26:35 +00:00
Dan Bond 3b3aa1f260
[NET-4103] ci: build s390x (#18067)
* ci: build s390x

* ci: test s390x

* ci: dev build s390x

* no GOOS

* ent only

* build: publish s390x

* fix syntax error

* fix syntax error again

* fix syntax error again x2

* test branch

* Move s390x conditionals to step level

* remove test branch

---------

Co-authored-by: emilymianeil <eneil@hashicorp.com>
2023-07-12 16:10:34 -07:00
Matt Keeler 1864874726
CI Updates (#17834)
* Ensure that git access to private repos uses the ELEVATED_GITHUB_TOKEN

* Bump the runner size for the protobuf generation check

This has failed previously when the runner process that communicates with GitHub gets starved causing the job to fail.
2023-06-22 10:56:45 -04:00
Michael Zalimeni 500dcb1f21
Set GOPRIVATE for all hashicorp repos in CI (#17817)
Consistently set GOPRIVATE to include all hashicorp repos, s.t. private
modules are successfully pulled in enterprise CI.
2023-06-21 11:26:27 -04:00
Curt Bushko 0c15748c5a
[core]: Pin github action workflows (#17695) 2023-06-13 13:00:55 -04:00
John Murret e9986e3774
ci:upload test results to datadog (#17206)
* WIP

* ci:upload test results to datadog

* fix use of envvar in expression

* getting correct permission in reusable-unit.yml

* getting correct permission in reusable-unit.yml

* fixing DATADOG_API_KEY envvar expresssion

* pass datadog-api-key

* removing type from datadog-api-key
2023-05-10 14:49:18 -06:00
Semir Patel e7bb8fdf15
Fix or disable pipeline breaking changes that made it into main in last day or so (#17130)
* Fix straggler from renaming Register->RegisterTypes

* somehow a lint failure got through previously

* Fix lint-consul-retry errors

* adding in fix for success jobs getting skipped. (#17132)

* Temporarily disable inmem backend conformance test to get green pipeline

* Another test needs disabling

---------

Co-authored-by: John Murret <john.murret@hashicorp.com>
2023-04-25 15:17:48 -05:00
John Murret 430df05e61
ci: Add success jobs. make go-test-enterprise conditional. build-distros and go-tests trigger on push to main and release branches (#16905)
* Add go-tests-success job and make go-test-enterprise conditional

* fixing lint-32bit reference

* fixing reference to -go-test-troubleshoot

* add all jobs that fan out.

* fixing success job to need set up

* add echo to success job

* adding success jobs to build-artifacts, build-distros, and frontend.

* changing the name of the job in verify ci to be consistent with other workflows

* enable go-tests, build-distros, and verify-ci to run on merge to main and release branches because they currently do not with just the pull_request trigger
2023-04-06 16:29:32 -06:00
John Murret ad3a68a040
temporarily disable macos-arm64 tests job in go-tests (#16898) 2023-04-05 17:10:31 -06:00
John Murret 3f74827593
always test oss and conditionally test enterprise (#16827) 2023-04-05 16:49:48 -06:00
John Murret dcb9da2eb2
ad arm64 testing (#16876) 2023-04-05 09:58:00 -06:00
John Murret afc8f978a2
ci: increase deep-copy and lint-enum jobs to use large runner as they hang in ENT (#16866)
* docs: add envoy to the proxycfg diagram (#16834)

* docs: add envoy to the proxycfg diagram

* increase dee-copy job to use large runner.  disable lint-enums on ENT

* set lint-enums to a large

* remove redunant installation of deep-copy

---------

Co-authored-by: cskh <hui.kang@hashicorp.com>
2023-04-04 09:15:45 -06:00
John Murret 99ba13b9fc
updating command to reflect the additional package exclusions in CircleCI (#16829) 2023-03-30 21:39:34 -06:00
John Murret aab9397bc2
ci: changes resulting from running on consul-enterprise (#16816)
* changes resulting from running on consul-enterprise

* removing comment line
2023-03-30 10:57:11 -06:00
John Murret a3973a82ae
Github Actions Migration - move go-tests workflows to GHA (#16761)
* go-tests workflow

* add test splitting to go-tests

* fix re-reun fails report path

* fix re-reun fails report path another place

* fixing tests for32bit and race

* use script file to generate runners

* fixing run path

* add checkout

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* passing runs-on

* setting up runs-on as a parameter to check-go-mod

* making on pull_request

* Update .github/scripts/rerun_fails_report.sh

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* make runs-on required

* removing go-version param that is not used.

* removing go-version param that is not used.

* Modify build-distros to use medium runners (#16773)

* go-tests workflow

* add test splitting to go-tests

* fix re-reun fails report path

* fix re-reun fails report path another place

* fixing tests for32bit and race

* use script file to generate runners

* fixing run path

* add checkout

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* passing runs-on

* setting up runs-on as a parameter to check-go-mod

* trying mediums

* adding in script

* fixing runs-on to be parameter

* fixing merge conflict

* changing to on push

* removing whitespace

* go-tests workflow

* add test splitting to go-tests

* fix re-reun fails report path

* fix re-reun fails report path another place

* fixing tests for32bit and race

* use script file to generate runners

* fixing run path

* add checkout

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* passing runs-on

* setting up runs-on as a parameter to check-go-mod

* changing back to on pull_request

---------

Co-authored-by: Dan Bond <danbond@protonmail.com>

* Github Actions Migration - move verify-ci workflows to GHA (#16777)

* add verify-ci workflow

* adding comment and changing to on pull request.

* changing to pull_requests

* changing to pull_request

* Apply suggestions from code review

Co-authored-by: Dan Bond <danbond@protonmail.com>

* [NET-3029] Migrate frontend to GHA (#16731)

* changing set up to a small

* using consuls own custom runner pool.

---------

Co-authored-by: Dan Bond <danbond@protonmail.com>
2023-03-28 15:29:27 -06:00