Day 82: A Password Removed and a Copy That Stayed Put

Day 82 of DevOps, Day 32 of Azure. Both tasks today returned a cheerful single word that meant less than it looked. Ansible said pong, which does not mean what most people think it means. Azure said success, which, for a small file, means finished, and for a large one, means scheduled. In both cases, the documentation is clear about the difference, and nothing in the output is. One Ansible task, one Azure task. Fix an inventory file so Ansible can reach three servers, then migrate a blob between containers. The tasks come from the KodeKloud Engineer platform. This is the first Ansible day. Days 82 to 93 are Ansible, which closes the gap I have been calling the clearest one on my CV since August. Task 1: An inventory with three passwords in it The task: modify the inventory so Ansible can reach all three app servers, and prove it works. Here is what the lab hands you: stapp01 ansible_ssh_pass=Ir0nM@n stapp02 ansible_ssh_pass=Am3ric@ stapp03 ansible_ssh_pass=BigGr33 Three passwords, in a file, in plain text. Step 1: Ansible's documentation states the rule at the point of use I went in thinking replacing those with keys was good practice. It is stronger than that. Of ansible_password, the inventory documentation says flatly: "never store this variable in plain text; always use a vault." And the same page sets the default expectation, which the lab file is working around rather than using: "By default, Ansible assumes you are using SSH keys to connect to remote machines", and "SSH keys are encouraged, but you can use password authentication if needed with the --ask-pass option." So the supplied inventory does two things wrong at once. It stores secrets in plain text, and it bypasses the connection method the tool assumes, in order to avoid setting that method up. Worth putting next to Day 80, nine days ago, where a Jenkins job had a server password typed into a build step three times over, readable by anyone with Configure on the job. Same decision, opposite outcome, and I do not think the difference is the engineer. Ansible tells you the rule in the variable's own description. Jenkins has no documentation page for the freestyle shell step at all. A tool that states the rule where you are working gets the rule followed. Step 2: The key, and the passphrase that matters more than it looks thor@jump-host ~$ ssh-keygen Generating public/private ed25519 key pair. Enter passphrase for "/home/thor/.ssh/id_ed25519" (empty for no passphrase): thor@jump-host ~$ ssh-copy-id tony@stapp01 Number of key(s) added: 1 Two things in that output are worth stopping on. Bare ssh-keygen with no flags gives ed25519. Which is the Day 74 correction landing: I used -t rsa -b 4096 on Day 71 and -t rsa -b 2048 on Day 74, and neither was a decision, because the manual gives ed25519 as the default type and 3072 as the RSA default size. So one was an upgrade and the other a downgrade from a default I had not looked up. Omitting both flags, as here, takes the newer default and stops the question arising. And the passphrase is empty, which turns out to matter more than it looks. In my notes for the next four days I wrote at that prompt instead, as a placeholder, and then showed a successful Ansible run. Those two are not interchangeable, and this is the day whose transcript records which one actually happened. Ansible's connection documentation states the limitation outright: with the default ssh connection plugin, Ansible "does not expose a channel to allow communication between the user and the ssh process to accept a password manually to decrypt an ssh key." A passphrase-protected key cannot be decrypted during an Ansible run. The documented way to use one is an agent, which the same page recommends: "The use of ssh-agent is highly recommended." None of my notes mention one. So the options are an empty passphrase, which makes the key file itself the only credential, or a passphrase plus a loaded agent, which is better and one more step. A passphrase with no agent does not work, and a placeholder in a write-up reads as an instruction, so it is worth replacing with the real prompt. Step 3: The inventory, and the group that isn't there stapp01 ansible_user=tony stapp02 ansible_user=steve stapp03 ansible_user=banner No group header. Ansible does not need one: "Even if you do not define any groups in your inventory file, Ansible creates two default groups: all and ungrouped", and "The all group contains every host." So those three lines are addressable individually, as all, and as ungrouped. Every other day in this batch adds a [group] header, and this task works without one. What the docs recommend thor@jump-host ~/ansible$ ansible stapp01 -i inventory -m ping stapp01 | SUCCESS => { "ansible_facts": { "discovered_interpreter_python": "/usr/bin/python3" }, "changed": false, "ping": "pong" } That is the right verification for this task, and it is worth knowing exactly what it proves, because the name misleads. The module's own documentation: "This is NOT ICMP ping, this is just a trivial test module that requires Python on the remote-node." And its narrow purpose and exactly what an inventory change needs: "It does not make sense in playbooks, but it is useful from /usr/bin/ansible to verify the ability to login and that a usable Python is configured." Two things, then. The login, which is what the key was for. And a usable Python, which is why the reply carries discovered_interpreter_python rather than nothing. That field is the second half of the check, made visible. Which means a host that is up, routable and listening on port 22 but has no Python fails this check, and a host unreachable by ICMP but reachable by SSH passes it. It is the correct tool for verifying an inventory and the wrong one for diagnosing a network. Day 8 of this series, back in July, was titled on exactly this point. The documentation has not changed and the trap has not stopped being worth stating. Task 2: A blob copied between containers The task: create a private container, migrate a file into it, and prove the content is identical. Step 1: The data never came to my machine az storage blob copy start --destination-container $DEST --destination-blob devops.txt \ --source-container $SRC --source-blob devops.txt Azure does this inside the storage service. The bytes move between containers without passing through the client at all, which matters three ways. Scale: identical command and identical cost for 33 bytes or 33 terabytes, with no local disk involved. Billing: a download followed by an upload pulls the data out of Azure and back in, billed as egress, where a same-account server-side copy transfers nothing externally. Speed: this returned immediately rather than streaming anything. The naive version, download then upload, gives the same result for a small file and is the wrong habit to carry into a real migration. Step 2: The word "success" is doing less work than it appears to "copy_id": "d356b703-fd74-43df-8b00-77e4f5540992", "copy_status": "success" copy start schedules the operation and hands back a copy_id. For a small blob inside one storage account it finishes before the response is written, so the status reads success straight away. A large blob, or a copy across accounts or regions, returns pending. Progress then lives on the destination blob rather than in the command's output: az storage blob show -c $DEST -n devops.txt --query "properties.copy.{Status:status,Progress:progress}" So a migration script cannot infer completion from what copy start returned. It has to poll until the status leaves pending, and handle failed and aborted. Which is the same shape as pong in the first half of today. Both commands answer with one encouraging word, both words cover less ground than they suggest, and in both cases the documentation says precisely what the word covers. Step 3: And the field names changed convention This is why I ran the command without a --query first: { "client_request_id": "...", "copy_id": "...", "copy_status": "success", "last_modified": "..." } Snake_case. Nearly every other az command returns camelCase, because the CLI serialises a typed SDK model. This one passes through a plain dictionary from the Python SDK, so the keys arrive as the SDK produced them. --query "{Status:copyStatus}" returns null; the correct form is --query copy_status. Fourth variant of one problem in this track: Variant Example Casing differs between resources diskSizeGB against diskSizeGb, Day 8 Field absent from one command contentMd5 on upload, Day 18 Sparse response object siteConfig on webapp create, Day 31 Different naming convention entirely copy_status, here One habit covers all four. Run an unfamiliar command with plain -o json once, then write the query. Guessing gives you null, and null is indistinguishable from an empty value. What the docs recommend The requirement was that both containers hold the file and the content is identical. Those are two claims and they need different evidence: Check Proves blob list on both containers The file exists in both Matching contentMd5 The service believes the content matches cmp on two downloads The bytes are identical The MD5 comparison is strong and still second-hand. It compares what Azure recorded at upload time, and a blob whose content settings were set wrongly would report a stale digest quite happily. cmp on the actual downloaded bytes is the only check that depends on nothing but the data. While doing that, a nice demonstration of something Day 18 described without an example. Azure reported Lu7zilatbGguzSz2Ecn5IQ== and md5sum printed 2eeef38a56ad6c682ecd2cf611c9f921. Those are the same 16 bytes: decode the base64 and you get that hex string exactly. Worth knowing before concluding a checksum mismatch means corruption. One last thing, on the task's title. It says "Synchronizing Containers" and the requirement is one file, which is the Day 14 pattern again: the title names a topic, the numbered requirements are the contract. If you did want the thing the title says, azcopy sync is it, because it compares source and destination and transfers only the differences. copy start-batch copies everything, every time. What I learned Two commands, two reassuring words, and neither means what it looks like. pong means Ansible logged in and found a working Python interpreter. It says nothing about the network, and a perfectly reachable host with no Python fails it. success means the copy was scheduled, and for anything big enough to matter it will say pending instead and expect you to go and look somewhere else. The useful part is that both tools document this precisely, in the place you would look if you thought to look. Which is the same thing the first half of today turned out to be about: Ansible puts "never store this variable in plain text" in the description of the variable itself, and that is why I replaced three passwords with a key without having to be clever about it. Nine days ago I wrote up a Jenkins job with a password in its build step three times over. The difference between these two days is not that I got better at security. It is that one tool told me the rule where I was standing. So here is the Day 82 question. Which of your green checks is reporting that something was scheduled rather than that it finished? Day 82 down. Eighteen to go.