Commit Graph

17441 Commits

Author SHA1 Message Date
Chris S. Kim 3178a3cb7a
Fix broken CI step (#13314) 2022-05-31 16:58:17 -04:00
Dhia Ayachi 1b779240ae
update gateway-services table with endpoints (#13217)
* update gateway-services table with endpoints

* fix failing test

* remove unneeded config in test

* rename "endpoint" to "destination"

* more endpoint renaming to destination in tests

* update isDestination based on service-defaults config entry creation

* use a 3 state kind to be able to set the kind to unknown (when neither a service or a destination exist)

* set unknown state to empty to avoid modifying alot of tests

* fix logic to set the kind correctly on CRUD

* fix failing tests

* add missing tests and fix service delete

* fix failing test

* Apply suggestions from code review

Co-authored-by: Dan Stough <dan.stough@hashicorp.com>

* fix a bug with kind and add relevant test

* fix compile error

* fix failing tests

* add kind to clone

* fix failing tests

* fix failing tests in catalog endpoint

* fix service dump test

* Apply suggestions from code review

Co-authored-by: Dan Stough <dan.stough@hashicorp.com>

* remove duplicate tests

* rename consts and fix kind when no destination is defined in the service-defaults.

* rename Kind to ServiceKind and change switch to use .(type)

Co-authored-by: Dan Stough <dan.stough@hashicorp.com>
2022-05-31 16:20:12 -04:00
Chris S. Kim f0a9b30174
Update repo to use go:embed (#10996)
Replace bindata packages with stdlib go:embed.
Modernize some uiserver code with newer interfaces introduced in go 1.16 (mainly working with fs.File instead of http.File.
Remove steps that are no longer used from our build files.
Add Github Action to detect differences in agent/uiserver/dist and verify that the files are correct (by compiling UI assets and comparing contents).
2022-05-31 15:33:56 -04:00
DanStough ee079eb4e3 chore(ci): add no-op jobs for branch protection rules 2022-05-31 15:24:22 -04:00
Riddhi Shah 1a901953e2
[OSS] Fix merge central config tests (#13309)
Setting the right enterprise meta to fix the merge central config tests.
Re-added the tests that were failing on the OSS to ENT merge.
2022-05-31 12:04:19 -07:00
freddygv 364758ef2f Use embedded SpiffeID for peered upstreams 2022-05-31 09:55:37 -06:00
freddygv c8edec0ab6 Remove intermediate representation of SPIFFE IDs
xDS only ever uses the string representation, so we can avoid passing
around connect.SpiffeIDService objects around.
2022-05-31 09:55:37 -06:00
freddygv 870e7c72d7 Return SPIFFE ID for connect proxies in PeerMeta
Proxies dialing exporting services need to know the SPIFFE ID of
services dialed so that the upstream's SANs can be validated.

This commit attaches the SPIFFE ID to all connect proxies exported over
the peering stream so that they are available to importing clusters.

The data in the SPIFFE ID cannot be re-constructed in peer clusters
because the partition of exported services is overwritten on imports.
2022-05-31 09:55:37 -06:00
Freddy 9427700270
[OSS] Add grpc endpoint to fetch a specific trust bundle (#13292)
Co-authored-by: R.B. Boyer <rb@hashicorp.com>
2022-05-31 09:54:40 -06:00
Chris S. Kim 8e24a56134
Refactor some functions for better enterprise use (#13280) 2022-05-30 09:46:55 -04:00
David Roca 357fe1c080
docs: Use .snap extension in API snapshot save/restore
Change the `.tgz` file extension in the snapshot save and restore
examples on /api-docs/snapshot to `.snap`.

This is consistent with the file extension used in other example
snapshot save and restore commands, as well as the default extension
used by the Consul Snapshot Agent.
2022-05-27 14:07:37 -07:00
Matt Keeler ead8e4a200
Fix race during proxy closing (#13283)
p.service is written to within the Serve method. The Serve method also waits for the stopChan to be closed.

The race was between Close being called on the proxy causing Close on the service which was written to around the same time in the Serve method.

The fix is to have Serve be responsible for closing p.service.
2022-05-27 16:52:03 -04:00
Evan Culver 9a13be3881
ci: add docker build smoke test (#13200) 2022-05-27 13:29:57 -07:00
cskh 64cfe245dd
CI: Verify built binaries in build job (#13221)
Co-authored-by: Evan Culver <eculver@hashicorp.com>
2022-05-27 14:50:41 -04:00
cskh 9e7e363627
CTIA-16: add tags to load test resources and run test on PR commit (#13258)
- retry destroy terraform resources
2022-05-27 14:49:39 -04:00
Matt Keeler 3795769729
Fix a flaky test (#13282)
At the end of this test we were trying to ensure that updating a service in the local state causes it to re-register the service with the config manager.

The config manager in the same method will also call RegisteredProxies to determine if any need to be removed. This portion of the test is not attempting to verify that behavior.

Because the test is only blocked waiting for the Register event before it can end and assert all the mock expectations were met, we may not see the call to RegisteredProxies. This is especially apparent when tests are run with the race detector.

As we don’t actually care if that method is executed before the end of the test we can simply transition from expecting it to be called exactly once to a 0 or 1 times assertion.
2022-05-27 13:25:08 -04:00
Chris S. Kim b2c4e8b2fe
Add build tag for oss (#13279) 2022-05-27 11:39:58 -04:00
Mathew Estafanous 428e32706e
Replace CLI command registry with a new pattern. (#12729) 2022-05-27 11:33:27 -04:00
Dan Upton 2427e38839
Enable servers to configure arbitrary proxies from the catalog (#13244)
OSS port of enterprise PR 1822

Includes the necessary changes to the `proxycfg` and `xds` packages to enable
Consul servers to configure arbitrary proxies using catalog data.

Broadly, `proxycfg.Manager` now has public methods for registering,
deregistering, and listing registered proxies — the existing local agent
state-sync behavior has been moved into a separate component that makes use of
these methods.

When an xDS session is started for a proxy service in the catalog, a goroutine
will be spawned to watch the service in the server's state store and
re-register it with the `proxycfg.Manager` whenever it is updated (and clean
it up when the client goes away).
2022-05-27 12:38:52 +01:00
alex fd7a403e11
monitor leadership in peering service (#13257)
Signed-off-by: acpana <8968914+acpana@users.noreply.github.com>

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
Co-authored-by: Freddy <freddygv@users.noreply.github.com>
2022-05-26 17:55:16 -07:00
Evan Culver bffb6d8ab8
Add latest changelog entries (#13276) 2022-05-26 16:14:02 -07:00
Riddhi Shah b6a4271c02
Termporarily disable validation of merge central config response (#13266)
Temporarily disabling the validation of merge central config response since
it is breaking OSS to ENT merging.
A follow up PR will patch the fixes.
2022-05-26 13:49:40 -07:00
Chris S. Kim 6d3bea7129
Add support for streaming CA roots to peers (#13260)
Sender watches for changes to CA roots and sends
them through the replication stream. Receiver saves
CA roots to tablePeeringTrustBundle
2022-05-26 15:24:09 -04:00
Jasmine W c052c17d20 Merge pull request #13239 from hashicorp/ui/bugfix/permissions-header
ui: Typography update for view-only Intentions
2022-05-26 14:47:49 -04:00
cskh e61e405fb1
Enable manual triggering of load test (#13068) 2022-05-26 14:18:14 -04:00
Riddhi Shah c78ee7d48f
Remove tests failing on ent (#13255)
Will follow up with the fixed version of these tests that passes in ent.
2022-05-26 10:17:59 -07:00
Michele Degges 407cd332ff
[CI-only] Support UBI images (#13232)
Co-authored-by: David Yu <dyu@hashicorp.com>
2022-05-26 09:49:47 -07:00
John Cowen 09c5bac102
Export top-level HCP Enabled go-template variable for UI (#13165)
* Update ui template data to export HCPEnabled at the top level
2022-05-26 17:23:56 +01:00
Jasmine W a2c20518c0 updates
readded %reset-typo and defined .consul-intention-view h2
2022-05-26 11:23:00 -04:00
DanStough 2e2c71d2f2 fix: multiple grpc/http2 services for ingress listeners 2022-05-26 10:43:58 -04:00
Jasmine W 439d9e7f65 removed %reset-typo 2022-05-25 19:17:17 -04:00
Jared Kirschner bf4d23a9e8
Merge pull request #13006 from hashicorp/docs/http-api-breakout-path-parameters
HTTP API Docs: Separate path parameters from query parameters
2022-05-25 18:20:40 -04:00
Jared Kirschner 49a7e7086c docs: split HTTP API params into sections by type
Path parameters, query parameters, and request body parameters are now shown in
separate sections rather than combined into one general parameters section.
This makes it much easier to understand quickly where a parameter should be
provided.
2022-05-25 14:45:47 -07:00
Riddhi Shah d8d8c8603e
Add support for merge-central-config query param (#13001)
Adds a new query param merge-central-config for use with the below endpoints:

/catalog/service/:service
/catalog/connect/:service
/health/service/:service
/health/connect/:service

If set on the request, the response will include a fully resolved service definition which is merged with the proxy-defaults/global and service-defaults/:service config entries (on-demand style). This is useful to view the full service definition for a mesh service (connect-proxy kind or gateway kind) which might not be merged before being written into the catalog (example: in case of services in the agentless model).
2022-05-25 13:20:17 -07:00
R.B. Boyer 31526139fd
remove a source of test panics (#13227) 2022-05-25 14:33:00 -05:00
R.B. Boyer a85b8a4705
api: ensure peering API endpoints do not use protobufs (#13204)
I noticed that the JSON api endpoints for peerings json encodes protobufs directly, rather than converting them into their `api` package equivalents before marshal/unmarshaling them.

I updated this and used `mog` to do the annoying part in the middle. 

Other changes:
- the status enum was converted into the friendlier string form of the enum for readability with tools like `curl`
- some of the `api` library functions were slightly modified to match other similar endpoints in UX (cc: @ndhanushkodi )
- peeringRead returns `nil` if not found
- partitions are NOT inferred from the agent's partition (matching 1.11-style logic)
2022-05-25 13:43:35 -05:00
Kyle Schochenmaier 72a1aea56c
update docs for single-dc-multi-k8s install (#13008)
* update docs for single-dc-multi-k8s install

Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
2022-05-25 11:34:56 -07:00
Michael Wilkerson e55a269601
Nia/docs 0.6.0 (#13107)
* updated docs
- added docs for start command
- deprecated running without a command
- added instructions for autocomplete setup

* addressed review comments

* addressed review comments

* addressed review comments

* docs/nia: Terraform Cloud agent support

- Add agent as a supported execution mode
- Add terraform_cloud_workspace configuration
- Deprecate existing terraform_version config

* license block docs

* added HCP Consul to compatibility

* added HCP instructions

* addressed review comments

* added new auto-retrieval behavior to license docs

* addressed review comments

* addressed review comments

* Apply suggestions from code review

* Apply suggestions from code review

* updated docs

* updated docs

* Apply suggestions from code review

Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

* fixed heading types

* fixed heading types

* update docs

* docs/nia: Add service registration configurations

* docs/nia: Style guide updates

* docs/nia: Update with beta docs feedback

* docs/nia: Update license config formatting

Other top-level blocks aren't included in the list of global config options,
so removed the liciense entry.

* docs/nia: Add auto-retrieval section to license page

* docs/nia: Separate column for HCP Consul support

* docs/nia: Compatiblity version upper bounds

* docs/nia: Fix broken links

* docs/nia: Style guide fixes

Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>

* Remove RequestId field from cts health api docs.

* docs/nia - Update CTS service id format (#13125)

* docs/nia: Convert Consul config to table format

* docs/nia: Add ACL token policy requirements

* update docs (#13174)

* docs/nia: Fix ca_path, key default, and some links

* docs/nia: Add CTS service address config

* Update website/content/docs/nia/cli/index.mdx

* docs/nia: update for 0.6 GA (#13191)



Co-authored-by: devarshishah3 <devarshishah3@gmail.com>
Co-authored-by: Michael Wilkerson <62034708+wilkermichael@users.noreply.github.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>

Co-authored-by: Melissa Kam <mkam@hashicorp.com>
Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com>
Co-authored-by: Melissa Kam <3768460+mkam@users.noreply.github.com>
Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com>
Co-authored-by: Derek Menteer <derek.menteer@hashicorp.com>
Co-authored-by: lornasong <lornasong@users.noreply.github.com>
Co-authored-by: hashi-derek <105233703+hashi-derek@users.noreply.github.com>
Co-authored-by: devarshishah3 <devarshishah3@gmail.com>
2022-05-25 14:23:43 -04:00
R.B. Boyer 1a8834e1c8
peering: replicate expected SNI, SPIFFE, and service protocol to peers (#13218)
The importing peer will need to know what SNI and SPIFFE name
corresponds to each exported service. Additionally it will need to know
at a high level the protocol in use (L4/L7) to generate the appropriate
connection pool and local metrics.

For replicated connect synthetic entities we edit the `Connect{}` part
of a `NodeService` to have a new section:

    {
      "PeerMeta": {
        "SNI": [
          "web.default.default.owt.external.183150d5-1033-3672-c426-c29205a576b8.consul"
        ],
        "SpiffeID": [
          "spiffe://183150d5-1033-3672-c426-c29205a576b8.consul/ns/default/dc/dc1/svc/web"
        ],
        "Protocol": "tcp"
      }
    }

This data is then replicated and saved as-is at the importing side. Both
SNI and SpiffeID are slices for now until I can be sure we don't need
them for how mesh gateways will ultimately work.
2022-05-25 12:37:44 -05:00
R.B. Boyer 35371ad697
build: re-add proto-tools dep to make proto (#13223) 2022-05-25 12:24:55 -05:00
Evan Culver a82b5f3933
update main to reflect 1.13.0-dev (#13192) 2022-05-25 09:06:36 -07:00
Evan Culver 2abccd78b4
Update CHANGELOG to mention removal of Envoy 1.17.4 and 1.18.6 (#13207) 2022-05-25 08:57:01 -07:00
R.B. Boyer be631ebdce
peering: disable requirement for mesh gateways initially (#13213) 2022-05-25 10:13:23 -05:00
Kyle Havlovitz 0ed9ff8ef7
Merge pull request #13143 from hashicorp/envoy-connection-limit
Add connection limit setting to service defaults
2022-05-25 07:48:50 -07:00
John Cowen a61e5cc08b
ui: Icon related fixups (#13183)
* ui: Use new icon-size and icon-color for popover-menus

* Remove the default currentColor plus add some more defaults

* Undo transparency overwrites now we don't need them

* Fixup discochain icons

* Undo a default icon rule for vert align

* Fixup expanded icon for meatball popovers

* Fixup intention permission labels/badges/icons

* Remove different res icon

* Remove icon resolutions
2022-05-25 14:28:42 +01:00
Kyle Havlovitz f2fbe8aec9 Fix proto lint errors after version bump 2022-05-24 18:44:54 -07:00
Michele Degges bfe7f0ad63
[CI-only] Update tagging for dev_tags (#13199)
Remove the hardcoded `-dev` suffix from dev_tags, which is causing tags to be in the format `1.12.0-dev-dev` instead of just `1.12.0-dev`. I'll clean up the old tags before making the dockerhub repo public, which will be available https://hub.docker.com/r/hashicorppreview/consul
2022-05-24 15:23:01 -07:00
Kyle Havlovitz dbed8ae10b Specify go_package explicitly 2022-05-24 10:22:53 -07:00
cskh 8712a088b1
fix: non-leader agents return 404 on Get Intention exact api (#13179)
* fix: non-leader agents return 404 on Get Intention exact api

- rpc call method appends extra error message, so change == to
  "Strings.Contains"

Co-authored-by: Chris S. Kim <ckim@hashicorp.com>
2022-05-24 13:21:15 -04:00
Kyle Havlovitz 4bc6c23357 Add connection limit setting to service defaults 2022-05-24 10:13:38 -07:00