Amazon CloudWatch Evidently API参考说明书

CloudWatch Evidently
API Reference
API Version 2021-02-01
CloudWatch Evidently: API Reference
Copyright © 2023 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.
Table of Contents
Welcome (1)
Actions (2)
BatchEvaluateFeature (3)
Request Syntax (3)
URI Request Parameters (3)
Request Body (3)
Response Syntax (4)
Response Elements (4)
Errors (4)
See Also (5)
CreateExperiment (6)
Request Syntax (6)
URI Request Parameters (6)
Request Body (7)
Response Syntax (9)
Response Elements (10)
Errors (10)
See Also (10)
CreateFeature (12)
Request Syntax (12)
URI Request Parameters (12)
Request Body (12)
Response Syntax (14)
Response Elements (15)
Errors (15)
See Also (15)
CreateLaunch (17)
Request Syntax (17)
URI Request Parameters (18)
Request Body (18)
Response Syntax (19)
Response Elements (20)
Errors (21)
Examples (21)
See Also (22)
CreateProject (23)
Request Syntax (23)
URI Request Parameters (23)
Request Body (23)
Response Syntax (24)
Response Elements (25)
Errors (25)
See Also (26)
CreateSegment (27)
Request Syntax (27)
URI Request Parameters (27)
Request Body (27)
Response Syntax (28)
Response Elements (28)
Errors (29)
Examples (29)
See Also (29)
DeleteExperiment (31)
Request Syntax (31)
Request Body (31)
Response Syntax (31)
Response Elements (31)
Errors (31)
See Also (32)
DeleteFeature (33)
Request Syntax (33)
URI Request Parameters (33)
Request Body (33)
Response Syntax (33)
Response Elements (33)
Errors (33)
See Also (34)
DeleteLaunch (35)
Request Syntax (35)
URI Request Parameters (35)
Request Body (35)
Response Syntax (35)
Response Elements (35)
Errors (35)
See Also (36)
DeleteProject (37)
Request Syntax (37)
URI Request Parameters (37)
Request Body (37)
Response Syntax (37)
Response Elements (37)
Errors (37)
See Also (38)
DeleteSegment (39)
Request Syntax (39)
URI Request Parameters (39)
Request Body (39)
Response Syntax (39)
Response Elements (39)
Errors (39)
See Also (40)
EvaluateFeature (41)
Request Syntax (41)
URI Request Parameters (41)
Request Body (42)
Response Syntax (42)
Response Elements (42)
Errors (43)
See Also (43)
GetExperiment (45)
Request Syntax (45)
URI Request Parameters (45)
Request Body (45)
Response Syntax (45)
Response Elements (46)
Errors (46)
See Also (47)
GetExperimentResults (48)
Request Syntax (48)
URI Request Parameters (48)
Response Syntax (50)
Response Elements (51)
Errors (51)
See Also (52)
GetFeature (53)
Request Syntax (53)
URI Request Parameters (53)
Request Body (53)
Response Syntax (53)
Response Elements (54)
Errors (54)
See Also (55)
GetLaunch (56)
Request Syntax (56)
URI Request Parameters (56)
Request Body (56)
Response Syntax (56)
Response Elements (57)
Errors (57)
See Also (58)
GetProject (59)
Request Syntax (59)
URI Request Parameters (59)
Request Body (59)
Response Syntax (59)
Response Elements (60)
Errors (60)
See Also (60)
GetSegment (62)
Request Syntax (62)
URI Request Parameters (62)
Request Body (62)
Response Syntax (62)
Response Elements (62)
Errors (63)
See Also (63)
ListExperiments (64)
Request Syntax (64)
URI Request Parameters (64)
Request Body (64)
Response Syntax (64)
Response Elements (65)
Errors (66)
See Also (66)
ListFeatures (67)
Request Syntax (67)
URI Request Parameters (67)
Request Body (67)
Response Syntax (67)
Response Elements (68)
Errors (68)
See Also (69)
ListLaunches (70)
Request Syntax (70)
URI Request Parameters (70)
Request Body (70)
Response Elements (71)
Errors (72)
See Also (72)
ListProjects (73)
Request Syntax (73)
URI Request Parameters (73)
Request Body (73)
Response Syntax (73)
Response Elements (74)
Errors (74)
See Also (74)
ListSegmentReferences (76)
Request Syntax (76)
URI Request Parameters (76)
Request Body (76)
Response Syntax (76)
Response Elements (77)
Errors (77)
See Also (78)
ListSegments (79)
Request Syntax (79)
URI Request Parameters (79)
Request Body (79)
Response Syntax (79)
Response Elements (80)
Errors (80)
See Also (80)
ListTagsForResource (82)
Request Syntax (82)
URI Request Parameters (82)
Request Body (82)
Response Syntax (82)
Response Elements (82)
Errors (83)
See Also (83)
PutProjectEvents (84)
Request Syntax (84)
URI Request Parameters (84)
Request Body (84)
Response Syntax (84)
Response Elements (85)
Errors (85)
See Also (85)
StartExperiment (87)
Request Syntax (87)
URI Request Parameters (87)
Request Body (87)
Response Syntax (87)
Response Elements (88)
Errors (88)
See Also (88)
StartLaunch (90)
Request Syntax (90)
URI Request Parameters (90)
Request Body (90)
Response Syntax (90)
Errors (91)
See Also (92)
StopExperiment (93)
Request Syntax (93)
URI Request Parameters (93)
Request Body (93)
Response Syntax (94)
Response Elements (94)
Errors (94)
See Also (95)
StopLaunch (96)
Request Syntax (96)
URI Request Parameters (96)
Request Body (96)
Response Syntax (97)
Response Elements (97)
Errors (97)
See Also (98)
TagResource (99)
Request Syntax (99)
URI Request Parameters (99)
Request Body (99)
Response Syntax (100)
Response Elements (100)
Errors (100)
See Also (100)
TestSegmentPattern (101)
Request Syntax (101)
URI Request Parameters (101)
Request Body (101)
Response Syntax (101)
Response Elements (101)
Errors (102)
See Also (102)
UntagResource (103)
Request Syntax (103)
URI Request Parameters (103)
Request Body (103)
Response Syntax (103)
Response Elements (103)
Errors (103)
See Also (104)
UpdateExperiment (105)
Request Syntax (105)
URI Request Parameters (105)
Request Body (106)
Response Syntax (107)
Response Elements (108)
Errors (108)
See Also (109)
UpdateFeature (110)
Request Syntax (110)
URI Request Parameters (110)
Request Body (110)
Response Syntax (112)
Response Elements (112)
See Also (113)
UpdateLaunch (114)
Request Syntax (114)
URI Request Parameters (114)
Request Body (115)
Response Syntax (116)
Response Elements (117)
Errors (117)
See Also (117)
UpdateProject (119)
Request Syntax (119)
URI Request Parameters (119)
Request Body (119)
Response Syntax (120)
Response Elements (120)
Errors (121)
See Also (121)
UpdateProjectDataDelivery (122)
Request Syntax (122)
URI Request Parameters (122)
Request Body (122)
Response Syntax (123)
Response Elements (123)
Errors (123)
See Also (124)
Data Types (125)
CloudWatchLogsDestination (127)
Contents (127)
See Also (127)
CloudWatchLogsDestinationConfig (128)
Contents (128)
See Also (128)
EvaluationRequest (129)
Contents (129)
See Also (129)
EvaluationResult (130)
Contents (130)
See Also (131)
EvaluationRule (132)
Contents (132)
See Also (132)
Event (133)
Contents (133)
See Also (133)
Experiment (134)
Contents (134)
See Also (137)
ExperimentExecution (138)
Contents (138)
See Also (138)
ExperimentReport (139)
Contents (139)
See Also (139)
ExperimentResultsData (140)
Contents (140)
See Also (140)
Contents (142)
See Also (142)
Feature (143)
Contents (143)
See Also (145)
FeatureSummary (146)
Contents (146)
See Also (147)
Launch (148)
Contents (148)
See Also (150)
LaunchExecution (151)
Contents (151)
See Also (151)
LaunchGroup (152)
Contents (152)
See Also (152)
LaunchGroupConfig (153)
Contents (153)
See Also (153)
MetricDefinition (155)
Contents (155)
See Also (156)
MetricDefinitionConfig (157)
Contents (157)
See Also (158)
MetricGoal (159)
Contents (159)
See Also (159)
MetricGoalConfig (160)
Contents (160)
See Also (160)
MetricMonitor (161)
Contents (161)
See Also (161)
MetricMonitorConfig (162)
Contents (162)
See Also (162)
OnlineAbConfig (163)
Contents (163)
See Also (163)
OnlineAbDefinition (164)
Contents (164)
See Also (164)
Project (165)
Contents (165)
See Also (167)
ProjectAppConfigResource (168)
Contents (168)
See Also (168)
ProjectAppConfigResourceConfig (169)
Contents (169)
See Also (169)
ProjectDataDelivery (170)
Contents (170)
See Also (170)
Contents (171)
See Also (171)
ProjectSummary (172)
Contents (172)
See Also (173)
PutProjectEventsResultEntry (175)
Contents (175)
See Also (175)
RefResource (176)
Contents (176)
See Also (177)
S3Destination (178)
Contents (178)
See Also (178)
S3DestinationConfig (179)
Contents (179)
See Also (179)
ScheduledSplit (180)
Contents (180)
See Also (180)
ScheduledSplitConfig (182)
Contents (182)
See Also (182)
ScheduledSplitsLaunchConfig (184)
Contents (184)
See Also (184)
ScheduledSplitsLaunchDefinition (185)
Contents (185)
See Also (185)
Segment (186)
Contents (186)
See Also (187)
SegmentOverride (188)
Contents (188)
See Also (188)
Treatment (189)
Contents (189)
See Also (189)
TreatmentConfig (190)
Contents (190)
See Also (190)
ValidationExceptionField (192)
Contents (192)
See Also (192)
VariableValue (193)
Contents (193)
See Also (193)
Variation (194)
Contents (194)
See Also (194)
VariationConfig (195)
Contents (195)
See Also (195)
Common Parameters (196)
Common Errors (198)
Welcome
You can use Amazon CloudWatch Evidently to safely validate new features by serving them to a specified percentage of your users while you roll out the feature. You can monitor the performance of the new
feature to help you decide when to ramp up traffic to your users. This helps you reduce risk and identify
unintended consequences before you fully launch the feature.
You can also conduct A/B experiments to make feature design decisions based on evidence and data. An
experiment can test as many as five variations at once. Evidently collects experiment data and analyzes it using statistical methods. It also provides clear recommendations about which variations perform better.
You can test both user-facing features and backend features.
This document was last published on December 4, 2023.
Actions
The following actions are supported:
•BatchEvaluateFeature (p. 3)
•CreateExperiment (p. 6)
•CreateFeature (p. 12)
•CreateLaunch (p. 17)
•CreateProject (p. 23)
•CreateSegment (p. 27)
•DeleteExperiment (p. 31)
•DeleteFeature (p. 33)
•DeleteLaunch (p. 35)
•DeleteProject (p. 37)
•DeleteSegment (p. 39)
•EvaluateFeature (p. 41)
•GetExperiment (p. 45)
•GetExperimentResults (p. 48)
•GetFeature (p. 53)
•GetLaunch (p. 56)
•GetProject (p. 59)
•GetSegment (p. 62)
•ListExperiments (p. 64)
•ListFeatures (p. 67)
•ListLaunches (p. 70)
•ListProjects (p. 73)
•ListSegmentReferences (p. 76)
•ListSegments (p. 79)
•ListTagsForResource (p. 82)
•PutProjectEvents (p. 84)
•StartExperiment (p. 87)
•StartLaunch (p. 90)
•StopExperiment (p. 93)
•StopLaunch (p. 96)
•TagResource (p. 99)
•TestSegmentPattern (p. 101)
•UntagResource (p. 103)
•UpdateExperiment (p. 105)
•UpdateFeature (p. 110)
•UpdateLaunch (p. 114)
•UpdateProject (p. 119)
•UpdateProjectDataDelivery (p. 122)
BatchEvaluateFeature
This operation assigns feature variation to user sessions. For each user session, you pass in an entityID
that represents the user. Evidently then checks the evaluation rules and assigns the variation.
The first rules that are evaluated are the override rules. If the user's entityID matches an override rule, the user is served the variation specified by that rule.
Next, if there is a launch of the feature, the user might be assigned to a variation in the launch. The
chance of this depends on the percentage of users that are allocated to that launch. If the user is
enrolled in the launch, the variation they are served depends on the allocation of the various feature
variations used for the launch.
If the user is not assigned to a launch, and there is an ongoing experiment for this feature, the user
might be assigned to a variation in the experiment. The chance of this depends on the percentage of
users that are allocated to that experiment. If the user is enrolled in the experiment, the variation they
are served depends on the allocation of the various feature variations used for the experiment.
If the user is not assigned to a launch or experiment, they are served the default variation.
Request Syntax
POST /projects/project/evaluations HTTP/1.1
Content-type: application/json
{
"requests": [
{
"entityId": "string",
"evaluationContext": "string",
"feature": "string"
}
]
}
URI Request Parameters
The request uses the following URI parameters.
project (p. 3)
The name or ARN of the project that contains the feature being evaluated.
Length Constraints: Minimum length of 0. Maximum length of 2048.
Pattern: (^[a-zA-Z0-9._-]*$)|(arn:[^:]*:[^:]*:[^:]*:[^:]*:project/[a-zA-
Z0-9._-]*)
Required: Yes
Request Body
The request accepts the following data in JSON format.
requests (p. 3)
An array of structures, where each structure assigns a feature variation to one user session.
Type: Array of EvaluationRequest (p. 129) objects
Array Members: Minimum number of 1 item. Maximum number of 20 items.
Required: Yes
Response Syntax
HTTP/1.1 200
Content-type: application/json
{
"results": [
{
"details": "string",
"entityId": "string",
"feature": "string",
"project": "string",
"reason": "string",
"value": { ... },
"variation": "string"
}
]
}
Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
results (p. 4)
An array of structures, where each structure displays the results of one feature evaluation assignment to one user session.
Type: Array of EvaluationResult (p. 130) objects
Errors
For information about the errors that are common to all actions, see Common Errors (p. 198). AccessDeniedException
You do not have sufficient permissions to perform this action.
HTTP Status Code: 403
ResourceNotFoundException
The request references a resource that does not exist.
HTTP Status Code: 404
ThrottlingException
The request was denied because of request throttling. Retry the request.
HTTP Status Code: 429
CloudWatch Evidently API Reference
See Also
ValidationException
The value of a parameter in the request caused an error.
HTTP Status Code: 400
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:•AWS Command Line Interface
•AWS SDK for .NET
•AWS SDK for C++
•AWS SDK for Go
•AWS SDK for Java V2
•AWS SDK for JavaScript V3
•AWS SDK for PHP V3
•AWS SDK for Python
•AWS SDK for Ruby V3
CreateExperiment
Creates an Evidently experiment. Before you create an experiment, you must create the feature to use for the experiment.
An experiment helps you make feature design decisions based on evidence and data. An experiment can
test as many as five variations at once. Evidently collects experiment data and analyzes it by statistical
methods, and provides clear recommendations about which variations perform better.
You can optionally specify a segment to have the experiment consider only certain audience types in
the experiment, such as using only user sessions from a certain location or who use a certain internet
browser.
Don't use this operation to update an existing experiment. Instead, use UpdateExperiment.
Request Syntax
POST /projects/project/experiments HTTP/1.1
Content-type: application/json
{
"description": "string",
"metricGoals": [
{
"desiredChange": "string",
"metricDefinition": {
"entityIdKey": "string",
"eventPattern": "string",
"name": "string",
"unitLabel": "string",
"valueKey": "string"
}
}
],
"name": "string",
"onlineAbConfig": {
"controlTreatmentName": "string",
"treatmentWeights": {
"string" : number
}
},
"randomizationSalt": "string",
"samplingRate": number,
"segment": "string",
"tags": {
"string" : "string"
},
"treatments": [
{
"description": "string",
"feature": "string",
"name": "string",
"variation": "string"
}
]
}
URI Request Parameters
The request uses the following URI parameters.
project (p. 6)
The name or ARN of the project that you want to create the new experiment in.
Length Constraints: Minimum length of 0. Maximum length of 2048.
Pattern: (^[a-zA-Z0-9._-]*$)|(arn:[^:]*:[^:]*:[^:]*:[^:]*:project/[a-zA-
Z0-9._-]*)
Required: Yes
Request Body
The request accepts the following data in JSON format.
description (p. 6)
An optional description of the experiment.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 160.
Pattern: .*
Required: No
metricGoals (p. 6)
An array of structures that defines the metrics used for the experiment, and whether a higher or lower value for each metric is the goal.
Type: Array of MetricGoalConfig (p. 160) objects
Array Members: Minimum number of 1 item. Maximum number of 3 items.
Required: Yes
name (p. 6)
A name for the new experiment.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 127.
Pattern: ^[-a-zA-Z0-9._]*$
Required: Yes
onlineAbConfig (p. 6)
A structure that contains the configuration of which variation to use as the "control" version. The
"control" version is used for comparison with other variations. This structure also specifies how much experiment traffic is allocated to each variation.
This parameter is required.
Type: OnlineAbConfig (p. 163) object
Required: No
randomizationSalt (p. 6)
When Evidently assigns a particular user session to an experiment, it must use a randomization ID to determine which variation the user session is served. This randomization ID is a combination of the entity ID and randomizationSalt. If you omit randomizationSalt, Evidently uses the experiment name as the randomizationSalt.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 127.
Pattern: .*
Required: No
samplingRate (p. 6)
The portion of the available audience that you want to allocate to this experiment, in thousandths of
a percent. The available audience is the total audience minus the audience that you have allocated to
overrides or current launches of this feature.
This is represented in thousandths of a percent. For example, specify 10,000 to allocate 10% of the available audience.
Type: Long
Valid Range: Minimum value of 0. Maximum value of 100000.
Required: No
segment (p. 6)
Specifies an audience segment to use in the experiment. When a segment is used in an experiment, only user sessions that match the segment pattern are used in the experiment.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 2048.
Pattern: (^[-a-zA-Z0-9._]*$)|(arn:[^:]*:[^:]*:[^:]*:[^:]*:segment/[-a-zA-
Z0-9._]*)
Required: No
tags (p. 6)
Assigns one or more tags (key-value pairs) to the experiment.
Tags can help you organize and categorize your resources. You can also use them to scope user
permissions by granting a user permission to access or change only resources with certain tag values.
Tags don't have any semantic meaning to AWS and are interpreted strictly as strings of characters.
You can associate as many as 50 tags with an experiment.
For more information, see Tagging AWS resources.
Type: String to string map
Key Length Constraints: Minimum length of 1. Maximum length of 128.
Key Pattern: ^(?!aws:)[a-zA-Z+-=._:/]+$
Value Length Constraints: Minimum length of 0. Maximum length of 256.
Required: No
treatments (p. 6)
An array of structures that describe the configuration of each feature variation used in the experiment.
Type: Array of TreatmentConfig (p. 190) objects
Array Members: Minimum number of 0 items. Maximum number of 5 items.
Required: Yes
Response Syntax
HTTP/1.1 200
Content-type: application/json
{
"experiment": {
"arn": "string",
"createdTime": number,
"description": "string",
"execution": {
"endedTime": number,
"startedTime": number
},
"lastUpdatedTime": number,
"metricGoals": [
{
"desiredChange": "string",
"metricDefinition": {
"entityIdKey": "string",
"eventPattern": "string",
"name": "string",
"unitLabel": "string",
"valueKey": "string"
}
}
],
"name": "string",
"onlineAbDefinition": {
"controlTreatmentName": "string",
"treatmentWeights": {
"string" : number
}
},
"project": "string",
"randomizationSalt": "string",
"samplingRate": number,
"schedule": {
"analysisCompleteTime": number
},
"segment": "string",
"status": "string",
"statusReason": "string",
"tags": {
"string" : "string"
},
"treatments": [
{
"description": "string",
"featureVariations": {
"string" : "string"
},
"name": "string"
}
],
"type": "string"
}
}
Response Elements
If the action is successful, the service sends back an HTTP 200 response.
The following data is returned in JSON format by the service.
experiment (p. 9)
A structure containing the configuration details of the experiment that you created.
Type: Experiment (p. 134) object
Errors
For information about the errors that are common to all actions, see Common Errors (p. 198). AccessDeniedException
You do not have sufficient permissions to perform this action.
HTTP Status Code: 403
ConflictException
A resource was in an inconsistent state during an update or a deletion.
HTTP Status Code: 409
ResourceNotFoundException
The request references a resource that does not exist.
HTTP Status Code: 404
ServiceQuotaExceededException
The request would cause a service quota to be exceeded.
HTTP Status Code: 402
ValidationException
The value of a parameter in the request caused an error.
HTTP Status Code: 400
See Also
For more information about using this API in one of the language-specific AWS SDKs, see the following:•AWS Command Line Interface
•AWS SDK for .NET •AWS SDK for C++•AWS SDK for Go
•AWS SDK for Java V2•AWS SDK for JavaScript V3•AWS SDK for PHP V3•AWS SDK for Python •AWS SDK for Ruby V3
CreateFeature
Creates an Evidently feature that you want to launch or test. You can define up to five variations of
a feature, and use these variations in your launches and experiments. A feature must be created in a
project. For information about creating a project, see CreateProject.
Don't use this operation to update an existing feature. Instead, use UpdateFeature.
Request Syntax
POST /projects/project/features HTTP/1.1
Content-type: application/json
{
"defaultVariation": "string",
"description": "string",
"entityOverrides": {
"string" : "string"
},
"evaluationStrategy": "string",
"name": "string",
"tags": {
"string" : "string"
},
"variations": [
{
"name": "string",
"value": { ... }
}
]
}
URI Request Parameters
The request uses the following URI parameters.
project (p. 12)
The name or ARN of the project that is to contain the new feature.
Length Constraints: Minimum length of 0. Maximum length of 2048.
Pattern: (^[a-zA-Z0-9._-]*$)|(arn:[^:]*:[^:]*:[^:]*:[^:]*:project/[a-zA-
Z0-9._-]*)
Required: Yes
Request Body
The request accepts the following data in JSON format.
defaultVariation (p. 12)
The name of the variation to use as the default variation. The default variation is served to users
who are not allocated to any ongoing launches or experiments of this feature.
This variation must also be listed in the variations structure.
If you omit defaultVariation, the first variation listed in the variations structure is used as the default variation.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 127.
Pattern: ^[-a-zA-Z0-9._]*$
Required: No
description (p. 12)
An optional description of the feature.
Type: String
Length Constraints: Minimum length of 0. Maximum length of 160.
Pattern: .*
Required: No
entityOverrides (p. 12)
Specify users that should always be served a specific variation of a feature. Each user is specified by a key-value pair . For each key, specify a user by entering their user ID, account ID, or some other identifier. For the value, specify the name of the variation that they are to be served.
This parameter is limited to 2500 overrides or a total of 40KB. The 40KB limit includes an overhead of 6 bytes per override.
Type: String to string map
Map Entries: Minimum number of 0 items. Maximum number of 20 items.
Key Length Constraints: Minimum length of 1. Maximum length of 512.
Key Pattern: .*
Value Length Constraints: Minimum length of 1. Maximum length of 127.
Value Pattern: ^[-a-zA-Z0-9._]*$
Required: No
evaluationStrategy (p. 12)
Specify ALL_RULES to activate the traffic allocation specified by any ongoing launches or
experiments. Specify DEFAULT_VARIATION to serve the default variation to all users instead.
Type: String
Valid Values: ALL_RULES | DEFAULT_VARIATION
Required: No
name (p. 12)
The name for the new feature.
Type: String
Length Constraints: Minimum length of 1. Maximum length of 127.。