devtop: Command line parser
Staring from this article I’m slowly working on devtop. I already described it at the end of the previous article, but it’s wise to repeat.
Devtop description
As operator I found that existing tooling fails with large number of unusual devices. If you have server with >100 virtual machines, and each virtual machines have one or more additional disk and a few network interfaces, it’s hard to find which VM cause stress on the server. Normal atop simply couldn’t cope with hundreds of tap/tun interfaces, multitude of block devices, etc. When I found that I have no proper tooling (as an operator) I wrote a shabby tools to do my job: they are calling blktop and ifstop. Both were written dirty and fast, and they does not follow the usual Python conventions (but they work as standalone binaries). They do their job, and normally I would say that’s enough. But those utilities are simple and have no big complications under the hood, so they are perfect candidates for rewriting in Rust for the sake of writing in Rust (I’m learning it!). I decide to combine them into a single utility: devtop, which I want to develop properly, according to Rust conventions, and through all aspects of packaging.
Command line parser
Command line is not a big deal for interactive utilities, but it need it anyway, so I start from parsing command line.
I looked at crates.io for command line parsing library for Rust and was overwhelmed. Too many of them.
I sorted them by numbers of download. The top library is ‘clapp’. Their documentation gives at least three ways to define command line arguments. I don’t like builder pattern, and I think that yaml config is overkill for a command line. I’ll try to use their macro-way to create a command line declaration (and parsing).
Even before I wrote a single line in Rust, I already got 9 crates to depend on. And release compilation time for ‘hello world’ took 34 seconds (on my old PC).
Output (release version) is 250k in size (182k after stripping), and I got two external runtime dependencies: libc and libgcc.
Sounds serious. I hope it worth it.
Command line design
I start from very basic command line: filters for device names and update interval. I’ll add more interesting things later.
My initial command line argument line look like this:
let matches = clap_app!(devtop =>
(@arg seconds: --("update-interval") -u +takes_value
default_value("2.0") "Delay between updates")
(@arg block_show: -b --("block-devices-show-filter")
+takes_value default_value(".+")
"Regexp for block devices to show")
(@arg net_show: -n --("net-devices-show-filter") +takes_value
default_value(".+") "Regexp for net devices to show" )
(@arg block_ignore: -B --("block-devices-ignore-filter")
+takes_value "Regexp to skip block devices")
(@arg net_ignore: -N --("net-devices-ignore-filter")
+takes_value "Regexpt to skip net devices" )
).get_matches();It on pair with python argparse boilerplate code, and I still need to validate all arguments. It less appealing then argparse even, because I need manually validate and convert parameter for update argument (from string to float) and I need to check if passed regexp is a valid regexp. I found no way to specify type of parameter in clap, so it should be done manually. If you knew the way, let me know.
Validating arguments
update argument need parameter with reasonable value: it should be float, it should be more then 0.01.
clap allow me to pass parameter validator to argument definition. It done by passing a closure (lambda, for python people) in curly brackets.
So I need to write a closure. Here example from the docs:
fn has_at(v: String) -> Result<(), String> {
if v.contains("@") { return Ok(()); }
Err(String::from("The value did not contain the required @ sigil"))
}It was very unusual for me to see empty Ok, but why not?
Signature for my closure is |u:String| -> Result<(), String>.
I’ll start by writing “always Ok” code. Two of my simple mistakes:
- validation snipped should be before help.
{|u:String| Ok()}is wrong,{|u:String| Ok(())}is right, because first brackets are for argument, and second to declare Unit (empty) element.
My working code (ignoring warnings):
(@arg seconds: --("update-interval") -u +takes_value
{|u:String| Ok(())}
default_value("2.0") "Delay between updates"
)Now I want to check if this argument is valid float or not. u.parse() should do the job, but now I need to pass ‘Ok()’ if I got something or pass through Err I got. I will search for unwrap_and or something like that.
My first candidate is map method for Option.
pub fn map<U, F>(self, f: F) -> Option<U> where F: FnOnce(T) -> UMaps an
Option<T>toOption<U>by applying a function to a contained value.
I’m on shaky ground here, as this is a moment of learning Rust.
Let me read the signature: map takes closure f such that it takes T (which I assume is only ‘T’ variant for Result) and return another Option.
My attempt:
{|u:String| u.parse().map(|x| ())}I got an error, but not where I expected it to be: type annotations required: cannot resolve `<_ as std::str::FromStr>::Err == std::string::String`
Uh. I played around but it didn’t buckle.
Ok, I’ll switch from a closure to a function.
(@arg seconds: --("update-interval") -u +takes_value
{is_float}
default_value("2.0") "Delay between updates"
)...
fn is_float(u: String) -> Result<(), String>{
Err("stub".to_string())
}
It works this way, my binary happily complaines: error: Invalid value for ‘ --update-interval <seconds>’: stub
Now I can focus just on function with a well-defined signature.
fn is_float(u: String) -> Result<(), String>{
let m: Result<f32, String> = u.parse();
Err("stub".to_string())
}The error become even more intriguing:
|
2 | let m: Result<f32, String> = u.parse();
| ^^^^^ expected struct `std::num::ParseFloatError`, found struct `std::string::String`
|
= note: expected type `std::num::ParseFloatError`
found type `std::string::String`This errors says that the second option for Result of parse is not a string, but std::num::ParseFloatError. I feel like i’m doing something wrong here, but…
fn is_float(u: String) -> Result<(), String>{
let m: Result<f32, std::num::ParseFloatError> = u.parse();
Err("stub".to_string())
}Ok, it has worked. Now I need to return ‘()’ if result is Ok, and Err (any string). I’ll come back to conversion latter.
fn is_float(u: String) -> Result<(), String>{
let m: Result<f32, std::num::ParseFloatError> = u.parse();
match m{
Ok(_) => Ok(()),
Err(_) => Err("stub".to_string())
}
}Yay! Hooray! It works and it complains on non-floats!
Now let’s focus on error message convention.
fn is_float(u: String) -> Result<(), String>{
let m: Result<f32, std::num::ParseFloatError> = u.parse();
match m{
Ok(_) => Ok(()),
Err(message) => Err(message.to_string())
}
}It works! I feel as I’m kindly guided by compiler. I barely understand exact type relationship here, but I done everything right.
fn is_float(u: String) -> Result<(), String>{
let m: Result<f32, std::num::ParseFloatError> = u.parse();
match m{
Ok(num) => if num < 0.01
{Err("Value is too small".to_string())}
else {Ok(())},
Err(message) => Err(message.to_string())
}
}I wasn’t able to find a match syntax for float relationships, so there is if.
But I found a bug: I can pass NaN as a parameter to my binary.
I decided to add a special case for NaN, but found that NaN is not a valid number fo Rust, it thinks that NaN is a variable number. I found std::f32::NAN, but Rust is insisting that I can’t do this:
warning: floating-point types cannot be used in patterns
--> src/main.rs:7:12
|
7 | Ok(std::f32::NAN) => Err("Invalid value".to_string()),
| ^^^^^^^^^^^^^
|
= note: #[warn(illegal_floating_point_literal_pattern)] on by default
= warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!
= note: for more information, see issue #41620 <https://github.com/rust-lang/rust/issues/41620>Falling back to ifs:
fn is_float(u: String) -> Result<(), String>{
let m: Result<f32, std::num::ParseFloatError> = u.parse();
match m{
Ok(num) => if num.is_nan() | num.is_infinite() {Err(u)}
else if num < 0.01
{Err("Value is too small".to_string())}
else {Ok(())},
Err(message) => Err(message.to_string())
}
}Note a clever trick to return ‘u’ as error in case of infintify or NaN:
It returns very sound errors to user:
./target/release/cmdparse --update-interval NaN
error: Invalid value for '--update-interval <seconds>': NaN
./target/release/cmdparse --update-interval inf
error: Invalid value for '--update-interval <seconds>': inf
./target/release/cmdparse --update-interval dsfsd
error: Invalid value for '--update-interval <seconds>': invalid float literal
./target/release/cmdparse --update-interval 0.001
error: Invalid value for '--update-interval <seconds>': Value is too smallSo far so good. But the function is too verbose for my taste. Can I make it concise? First, I can’t just move type ‘ascription’ to match sentence.
error[E0658]: type ascription is experimental (see issue #23416)
--> src/main.rs:5:11
|
5 | match u.parse():Result<f32, std::num::ParseFloatError> {
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^error: aborting due to previous errorFor more information about this error, try `rustc --explain E0658`.
I looked into String help page and found a way to specify type for parse function: parse::<u32>().
Good, I was able to simplify parse call and get rid of a variable:
fn is_float(u: String) -> Result<(), String>{
match u.parse::<f32>() {
Ok(num) => if num.is_nan() | num.is_infinite() {Err(u)}
else if num < 0.01{Err("Value is too small".to_string())}
else {Ok(())},
Err(message) => Err(message.to_string())
}
}
Can I squish nan and infinity things to match? I think it’s called guards…
How cow! It worked! As planned!
fn is_float(u: String) -> Result<(), String>{
match u.parse::<f32>() {
Ok(num) if num.is_nan() | num.is_infinite() => {Err(u)},
Ok(num) if num < 0.01 => {Err("too small".to_string())},
Ok(_) => {Ok(())},
Err(message) => Err(message.to_string())
}
}Moreover, I found that f32 have a very funny function which sounds just right for me: is_normal(). It will reject 0, but I reject 0 anyway, so:
Ok(num) if !num.is_normal() => {Err(u)},It will now complain on ‘0’ without additional information, so I change order of matches: first to check if it too small, and then to check if it’s normal.
My function is large enough to be a function, but fo educational reasons I want to make it a closure. It was simple, just copypaste with minimal changes. Resulting code is good for me, but I have doubts about formatting:
let matches = clap_app!(devtop =>
(@arg seconds: --("update-interval") -u +takes_value
{|u| match u.parse::<f32>() {
Ok(num) if num < 0.01 => {Err("too small".to_string())},
Ok(num) if !num.is_normal() => {Err(u)},
Ok(_) => {Ok(())},
Err(message) => Err(message.to_string())
}}
default_value("2.0") "Delay between updates"
)
....Validating regexps
I got four arguments with regexps. It would be nice to reject them if they are not a valid regexp. Moreover, it will help me to train to write the same type functions as those above.
I added module regex to my dependencies… 10 more modules. Now I look like a real application with tons of dependencies.
Because all regexp validation will be the same, I will keep them as a separate function.
The function was trivial to write (this time):
fn is_good_regexp(re: String) -> Result<(), String>{
match regex::Regex::new(&re) {
Ok(_) => Ok(()),
Err(error) => Err(error.to_string())
}
}And error message was beautiful (though it took me some time to find and invalid expression for regexp):
/target/release/cmdparse -n '[a'
error: Invalid value for '--net-devices-show-filter <net_show>': regex parse error:
[a
^
error: unclosed character classBut the stripped size of my binary has grown to 1.6 Mb. Should I start to worry?
Today’s finale
It was a big lesson for me, with a microscopic yield in terms of useful code. My current code is too small to move from ‘prg/rust’ to a normal git. I’ll continue doing this, with, may be, occasional glances on the Rust Book.
