Observe Amazon OpenSearch Service configuration modifications extra simply with new visibility enhancements

Amazon OpenSearch Service presents a number of area configuration settings to satisfy your workload-specific necessities. As a part of commonplace service operations, you could be required to replace these configuration settings frequently. Not too long ago, Amazon OpenSearch Service launched visibility enhancements that permit you to observe configuration modifications extra successfully. We’ve launched granular and extra descriptive configuration statuses that allow you to arrange alarms and use them in automation to reduce guide monitoring.

We advocate that you simply reap the benefits of these visibility enhancements in your purposes. These modifications are backward appropriate, and in case your automations depend on the legacy processing parameter to find out configuration change standing, then they need to nonetheless proceed to work with none disruption. To simplify monitoring of a number of in-flight configuration change requests, Amazon OpenSearch Service permits configuration request solely when Area Processing Standing is Lively. Further particulars are in part ‘Single configuration change at a time’.

Resolution overview

Earlier, configuration change standing visibility was obtainable via processing parameters within the OpenSearch Service APIs (Software Programming Interface), and as a Area Standing subject within the OpenSearch Service console. We’ve got now launched the next modifications to enhance the configuration replace expertise:

• Launched two new parameters, DomainProcessingStatus and ConfigChangeStatus, within the API responses. Equally, added Area Processing Standing and Configuration Change Standing fields within the console. These modifications present higher visibility via a number of, intuitive statuses. Earlier statuses have been restricted to solely two values: Lively and Processing.
• Capability to simply examine lively and in-flight configurations for readability. Earlier, it required a number of steps.
• Amazon OpenSearch Service has now adopted the method of permitting a single configuration change request at a time. There isn’t a restrict on the variety of area configuration modifications you’ll be able to bundle in a single request. Nevertheless, you’ll be able to submit the subsequent configuration request when the earlier request is full and the area processing standing turns into Lively. This enchancment streamlines configuration updates and addresses earlier challenges of monitoring a number of, in-flight configuration change requests.
• Capability to cancel a change request in case of a validation failure. Beforehand, when situations have been unavailable, domains remained in processing state. Now, upon encountering any validation failure, you’ll be able to cancel the change request and retry after a while.
• Area processing standing turns to Lively solely after all of the background actions, together with shard motion is full. This implies that you may confidently use newly launched statuses in your automation scripts while not having to deduce if all the inner processes, comparable to knowledge motion, are full.

How do you get granular particulars to trace the configuration replace standing?

As a part of current enhancements, Amazon OpenSearch Service launched DomainProcessingStatus and ConfigChangeStatus parameters within the APIs together with the respective Area Processing Standing and Configuration Change Standing fields within the console. You may depend on these statuses to get correct and constant data throughout completely different configuration change situations, like when configuration modifications contain blue/inexperienced operations or with out blue/inexperienced operations, and when configuration modifications are triggered by the operator or by the OpenSearch Service. Allow us to discover these enhanced visibility experiences.

1. Area processing standing visibility: You may observe the staus of domain-level configuration modifications via the Area Processing Standing subject within the console. Equally, API responses embrace the DomainProcessingStatus parameter. The values and a quick description are supplied within the following particulars:
   1. Lively: No configuration change is in progress. You may submit a brand new configuration change request.
   2. Creating: New area creation is in progress.
   3. Modifying: This standing signifies that a number of configuration modifications, comparable to new knowledge node addition, Amazon Elastic Block Retailer (Amazon EBS) GP3 storage provisioning, or organising KMS keys, are in progress. In different phrases, modifications made via the UpdateDomainConfig API, set the standing to modifying. The ‘Modifying’ standing additionally covers conditions the place domains require shard motion to finish configuration modifications. Word: For backward compatibility, we now have stored the habits of the processing parameter unchanged within the API responses, and it’s set to false as quickly because the core configuration modifications are full, with out ready for shard motion completion.
   4. Upgrading Engine Model: Engine model upgrades are in progress, comparable to from Elasticsearch model 7.9 to OpenSearch model 1.0.
   5. Updating Service Software program: This standing refers to configuration modifications associated to service software program updates.
   6. Deleting: Area deletion is progressing.
   7. Remoted: This represents domains which can be suspended because of quite a lot of causes, comparable to account-related billing points or domains that aren’t compliant with vital safety patch updates.
2. Configuration change standing visibility: Configuration modifications might be initiated by the person (e.g., new knowledge node addition, occasion kind change) or by the Service (e.g., AutoTune and obligatory service software program updates). You will discover the newest standing particulars via Configuration Change Standing subject within the console, and thru the ConfigChangeStatus parameter in API responses. Under are the values and a quick description:
   1. Pending: Signifies {that a} configuration change request has been submitted.
   2. Initializing: Service is initializing a configuration change request.
   3. Validating: Service is validating the requested modifications and assets required.
   4. Validation Failed: Requested modifications failed validation. At this level, no configuration modifications are utilized. Some attainable validation failures might be the presence of crimson indices within the area, unavailability of a selected occasion kind, and low disk area. Right here is an inventory of potential validation failures. Throughout a validation failure occasion, you’ll be able to cancel, retry, or edit configuration modifications.
   5. Awaiting person inputs: Situations the place person could possibly repair validation errors comparable to invalid KMS key. At this standing, person can edit the configuration modifications.
   6. Making use of modifications: Service is making use of requested configuration modifications.
   7. Cancelled: Throughout validation failed standing, you’ll be able to both click on on the Cancel button within the console or name the CancelDomainConfigChange API. All of the utilized modifications that have been a part of the change request will probably be rolled again.
   8. Accomplished: Requested configuration modifications have been efficiently accomplished.

Console enhancements

The Amazon OpenSearch Service console presents enhanced visibility to trace configuration change progress. Under are a couple of screenshots to present you an concept of those enhancements.

• Amazon OpenSearch Service console gives Area Processing Standing, Configuration Change Standing, and Change ID fields. Word: To know the change particulars related to the Change ID, you should use the DescribeDomainChangeProgress API.

• Configuration change abstract. To see a facet by facet comparability of your lively configurations and requested modifications, on the area element web page, navigate to the cluster configuration tab, scroll all the way down to the configuration change abstract part. Pending Adjustments subject exhibits the standing of the pending properties at the moment and doesn’t embrace modifications which were utilized. You can even get comparable particulars from the DescribeDomain and DescribeDomainConfig APIs via theModifyingProperties parameter.

Cancelling throughout validation failure. Within the under screenshots, you’ll be able to see a brand new choice to cancel a change request when a configuration change request fails validations. For instance, once you encounter SubnetNotFound error, you should use the Cancel request button to roll again to the earlier lively configuration, repair the problem after which retry the configuration replace.

Single configuration change at a time

Beforehand, it was not easy to trace the success and failure of particular person change requests, when a number of requests have been made. To offer a simplified expertise, OpenSearch Service now limits you to solely a single change request at a time. In a single configuration change request, you’ll be able to bundle a number of modifications without delay. As soon as a configuration change request is submitted, it should be accomplished earlier than you’ll be able to request the subsequent configuration change via the console, or via the UpdateDomainConfig API. This simplified expertise makes it simpler to maintain observe of modifications which were requested and their most up-to-date standing. In case your automation is written to name configuration change replace APIs a number of instances, then it ought to be up to date to group a number of configuration modifications in a single replace name, or look ahead to particular person updates to finish earlier than you submit the subsequent configuration change. You may replace area configuration when area processing standing turns into lively. For an inventory of modifications that may want a blue/inexperienced deployment, please see right here.

The under screenshot exhibits an instance alert on the ‘Edit area’ web page informing the person that one other change or replace is in progress. OpenSearch Service now not permits you to submit new configuration replace requests, and the ‘Apply change’ button is disabled till the change in progress is accomplished.

API modifications

You need to use the DescribeDomain, DescribeDomainChangeProgress, and DescribeDomainConfig APIs to get detailed configuration replace statuses. As well as, you should use CancelDomainConfigChange to cancel the change request within the occasion of a validation failures. You may refer Amazon OpenSearch Service API documentation right here.

Conclusion

On this submit, we confirmed you the way to get granular details about a configuration replace request. These newly launched modifications will aid you achieve higher visibility into the progress of configuration change requests, and simply distinguish between utilized modifications and pending ones. You have to make sure that the DomainProcessingStatus processing standing worth is Lively earlier than submitting configuration change requests. The power to cancel modifications within the occasion of validation failures provides you higher management in getting your area out of processing state in a self-service method. Go to product documentation to be taught extra.

Concerning the Authors

Siddhant Gupta is a Sr. Technical Product Supervisor at Amazon Internet Companies based mostly in Hyderabad, India. Siddhant has been with Amazon for over six years and is presently working with the OpenSearch Service staff, serving to with new area launches, pricing technique, and bringing EC2 and EBS improvements to OpenSearch Service clients. He’s captivated with analytics and machine studying. In his free time, he loves touring, health actions, spending time along with his household and studying non-fiction books.

Deniz Ercelebi is a Sr. UX Designer at Amazon OpenSearch Service. In her function she contributes to the creation, implementation, and profitable supply of design options for complicated issues. Her private drive is fueled by a ardour for person expertise, a dedication to customer-centric options, and a agency perception in collaborative innovation.

Shashank Gupta is a Sr. Software program Developer at Amazon OpenSearch Service, specializing within the enhancement of the Managed service facet of the platform. His major focus is on optimizing the managed expertise, spanning from the console to APIs and useful resource provisioning in an environment friendly method. With a devoted dedication to innovation, Shashank goals to raise the general buyer expertise by introducing ingenious options throughout the service.