I have had the fun of playing with Application Services previously known as Application Director or AppD. There has been lots of improvements done over the last 12 months one of the greatest ones for me is the resume feature that was delivered in 6.1. This was a god send specially when you have 4 hour deployments with multi tiered multi external services deployment and it fails 3 hours in due to the database service being down. The resume saved so much time in development of these deployments.
The next biggest gripe which has not been addressed is the ability to do an assisted tear down on a failed deployment. almost all my deployments configure firewalls/load balancers/account creation/password creation and storage/database creation/ip address allocation all of which need to be cleaned up on a tear down. Now when a deployment failed 3 quarters through and needs to be torn down there is only a quick tear down available. meaning allot of manual work to clean up what has been created.
This started me on the path of playing with the Application Service database. This is a PostgreSQL database located on the application services appliance.
I have a large application deployment which has failed 2 hours in and can not be resumed it has created many artifacts that will need to manually torn down because we can not run an assisted teardown on a failed applications service deployment. This manual teardown is fiddly, multiple domains, multiple firewalls, databases, allocated ip addresses and more, not only does this take time we have to trust it was completely torn down correctly and nothing was missed.
So I thought why not force appd to think it was successful…
To be able to connect to this database from an external source it will require some config changes. The details provided in THIS KB will help you configure the local postresql database a side note and this had me stumped for a while. the /home/darwin/pgsql/data/postgres.conf has 2 listen_ipaddresses entries and both must be changed.
Background on how it fits together in the database:
Each attempt to deploy or to resume a deployment creates a deployment task. below image is an example from the Application Services deployment summary view, the deployment id that is highlighted in the deployment name – this is the key to the deployment table in the database.
Each deployment task is related to its own instances of deployment nodes, which have one or more deployment node instances. Each deployment node instance can have many deployment node tasks. This portion of the execution plan for the first of the above deployment tasks shows these relationships:
Each of the objects described above has its own table in the database, and they can be linked together by performing joins on the appropriate id values. Some fields that are of particular interest to us are highlighted:
The state of tasks and deployments are represented through a code. The below tables which are available in the database to view are listed below.
|1||Task Scheduled||The deployment task has been scheduled.|
|2||Task In Progress||The deployment task is in progress.|
|3||Deployment Success||The deployment is in a successful state.|
|4||Deployment With Issues||One or more deployment tasks for the deployment has failed.|
|5||Deployment Torn down||The deployment is torn down|
|1||Scheduled||The deployment task has been scheduled.|
|2||In Progress||The deployment task is in progress.|
|3||Success||The deployment task completed successfully.|
|4||Failed||The deployment task failed|
|5||Stopped||The deployment task stopped|
|6||Stopping||The deployment task is being stopped|
|2||Not Started||The task has not started|
|3||Starting||The task is starting|
|4||Running||The task is running|
|5||Rebooting||VM is rebooting|
|6||Completed||The task has completed|
|7||Stopped||The task has is stopped|
|9||Waiting||The task is waiting on other task(s) to complete|
|10||Failed||The task has failed|
|11||Did Not Run||The task did not run|
|12||Initializing||The task is initializing|
|13||Provisioning VM||VM is being provisioned|
|14||VM Provisioned||VM is provisioned|
|15||Starting VM||VM is starting|
|16||VM Started||VM is started|
|17||Stopping VM||VM is stopping|
|18||VM Stopped||VM is stopped|
|19||Deprovisioning VM||VM is being deprovisioned|
|20||VM Deprovisioned||VM is deprovisioned|
|23||Not started on server||Not started on server|
So now we have a failed deployment we need mark as a success so we need to change the state. We could just mark the whole deployment as a success.
Using a tool like pgadmin we can connect in and just change the deployment_state_type_id in the deployment_task table. with a 3 and save it. (This is the bare minimum to allow an assisted teardown. (Note – will need to destroy all external service vms if used in the deployment)
below is a sql command that could be run, where ID is the task id of the deployment.
SET run_state_type_id = 6 -- Completed
,deployment_task_state_type_id = 3 -- Success
WHERE id = 9999
This will mark the deployment as complete and allow us to run an assisted teardown.. wooohoo
But say we had a deployment failure right at the end because of something dumb. everything is fine but the deployment failed on the very last step. We can not only mark the deployment as a success but also the failed task.
we can run this command to update the node tasks.
SET run_state_type_id = 6 -- Completed
,log_description = 'Log is missing because the task was manually marked as completed'
WHERE id = 999999
not when looking at the deployment overview everything will look as a success.