Skip to content

phd: collect core when killing non-booting guest #1079

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iximeow merged 1 commit into from
Mar 12, 2026
+47 −2
Merged

phd: collect core when killing non-booting guest #1079

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion phd-tests/framework/src/test_vm/mod.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -887,7 +887,15 @@ impl TestVm {
.instrument(info_span!("wait_to_boot"));

match timeout(timeout_duration, boot).await {
Err(_) => anyhow::bail!("timed out while waiting to boot"),
Err(_) => {
error!(
"Guest did not boot after {}ms! Collecting core..",
timeout_duration.as_millis()
);
let proc = self.server.as_ref().unwrap();
proc.core();
anyhow::bail!("timed out while waiting to boot")
Copy link
Member

@hawkw hawkw Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we ought to stuff the core's path in this error so that it gets printed in the test failure as well as in its logs?

Copy link
Member Author

@iximeow iximeow Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think i've got it so that the warn!("core written to {}", core_path); would be right above this in the rendered buildomat output, so it should be pretty easy to notice if you've gotta look at the logs.. it stubbornly does not want to do the thing though so i guess we'll see?

anyway i'm mostly thinking that until very recently most timed out while waiting to boot really meant that i did something funky with the guest test image or adapter. there the core wouldn't have been nearly as useful as looking at the guest's serial history, so i don't wanna nudge in a misleading direction.

}
Ok(inner) => {
inner.context("executing guest login sequence")?;
}
Expand Down
39 changes: 38 additions & 1 deletion phd-tests/framework/src/test_vm/server.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,12 @@ use std::{
fmt::Debug,
net::{SocketAddr, SocketAddrV4},
os::unix::process::CommandExt,
time::SystemTime,
};

use anyhow::Result;
use camino::{Utf8Path, Utf8PathBuf};
use tracing::{debug, info};
use tracing::{debug, info, warn};

use crate::log_config::LogConfig;

Expand Down Expand Up @@ -44,6 +45,7 @@ pub struct ServerProcessParameters<'a> {
pub struct PropolisServer {
server: Option<std::process::Child>,
address: SocketAddrV4,
output_dir: Utf8PathBuf,
}

impl PropolisServer {
Expand Down Expand Up @@ -117,6 +119,9 @@ impl PropolisServer {
let server = PropolisServer {
server: Some(server_cmd.spawn()?),
address: server_addr,
// Stash the same output directory in case the framework has to
// write any files on behalf of the test run.
output_dir: output_dir.to_owned(),
};

info!(
Expand All @@ -130,6 +135,38 @@ impl PropolisServer {
self.address
}

/// Collect a core of this server process, placing it in the same output
/// directory as other artifacts of this test.
pub(super) fn core(&self) {
let Some(server_proc) = self.server.as_ref() else {
warn!("Tried to produce a core without a propolis-server?");
return;
};

let core_name = format!(
"core-{}",
SystemTime::now()
.duration_since(SystemTime::UNIX_EPOCH)
.expect("Time is gone, the song is over")
.as_millis()
);
let core_path = self.output_dir.join(core_name);

std::process::Command::new("pfexec")
.args([
"gcore".as_ref(),
"-o".as_ref(),
core_path.as_os_str(),
server_proc.id().to_string().as_ref(),
])
.spawn()
.expect("can try to gcore a process")
.wait()
.expect("can gcore a propolis-server we spawned");

warn!("core written to {}", core_path);
}

/// Kills this server process if it hasn't been killed already.
pub(super) fn kill(&mut self) {
let Some(mut server) = self.server.take() else {
Expand Down
Loading