We have a web application. The application is for companies in the construction industry to manage their delivery tickets. The application is a modern web application built on the MEAN stack with a well designed RESTful API. The API is consumed by the in-house built web portal and the Android/iOS app. The API is also open to any third-party program for integration.
Everything looks good. More and more customers signed up to use this application. One of the most requested functions from customers is to export the tickets in a certain format and import into their accounting system for billing purposes.
We thought the open API is enough. But challenges here are:
- Almost every customer uses a different accounting software. Some of them prefer XML. Some of them prefer CSV. Very few like JSON.
- Most of those accounting systems are still on-premise software installed in their local network.
- Even some customers are using the same accounting software, they could ask for a different format with different data fields.
- Most customers don’t have in-house developers
We decide to build a CLI program to fill this gap. Even though we could build every export format into the customer web portal with configurations, but we always find it is not flexible enough to meet customer’s all kind of requirements with the export format. Also, some customers want to automate this process in their local network. A CLI program is perfect for this scenario. With the CLI, we could write a script to automate the process and customize the export data format with the customization settings saved locally.
- What is Command-line Interface(CLI)? See the wikipedia.
Some great examples are:
- Why CLI?
Programs with command-line interfaces are generally easier to automate via scripting.
- Why .NET Core?
Obviously .NET Core is not the only option to build a CLI. And it is not even the most popular one at this moment for sure. Python or Node.js could be the better option if you are familiar with them. But compare with .NET, .NET Core at least has met one of the most important requirements: cross platform.
- Windows 10
- Visual Studio 2017 Community Edition
- .NET Core 2.1
- Create a new project choose [Console App (.NET Core)]
- Edit Project File by right clicking the project
In the project file, change
LangVersion 7.1 enables async main , you can also use
RuntimeIdentifier enables publishing self-contained deployments.
AssemblyName specifies the output exe file name.
We use the appsetting.json file to save the configuration in .NET Core.
- Generic Host
.Net Core 2 introduced the Generic Host.The host is responsible for app startup and lifetime management.
A very good document about the Generic Host here.
By using the generic host introduced in .NET Core recent version, it will make it much easier to configure the logging, dependency injection etc.
Serilog is one of the most popular logging libraries for .NET Core.
We use the appsetting.json to configure the serilog
Putting the configuration in the setting file rather than in the code makes it easier to change the logging behavior without modifying the code. The configuration above uses a rolling file for each day.
ConfigureServices, it enables the
ILoggerinterface available for injection by providing the
SerilogLoggerProvider as the Logging provider.
- Dependency Injection
ConfigureServices passes the
IServiceCollection, which can be used to configure the dependency injections. For example,
- Command Line parser
One of the challenges of writing a CLI program is how to parse the command line. We can always write the parser ourselves from scratch but this is too time consuming even though it is a very interesting task for a coder. There are a couple of popular command line parser libraries available for .NET Core:
The dotnet/command-line-api has a very interesting story behind. And it is somehow related to natemcmaster/CommandLineUtils. You can read the story here. Hopefully it can be included in .NET Core someday. Read through it’s document is very helpful to develop a CLI program.
Also, understand how the windows command line involves is very helpful and interesting.
We eventually chose to use https://github.com/natemcmaster/CommandLineUtils, We like it’s
This line of code will use the class
iStradaCmd as the main/root command to start the command line parser and execution with the dependency injection built in. It works perfectly with the generic host.
There are three important concepts in this library: command, option and argument.
Those concepts came from the unix world Command line structure
Option is a name/value pair, and argument is just value. That simply means, option can show up in the command line in different places because the name can tell what option it is. The argument , on the other hand, has no name, so it has to appear in a certain place in the command line.
The CommandLineUtils library supports the sub command. For example,
git commit -m “initial”
git is the main/root command, commit is the sub command.
The CommandLineUtils library uses the attributes to map the command/sub command to class, and map the option and argument to class property. This bridges between parse results and functionality. The type of the command class property can be
String, Boolean, Byte, Int16, Int32, Int64, UInt16, UInt32, UInt64, Float, Double, Uri, DateTime, DateTimeOffset, TimeSpan, Array, Enum, HashSet, List, Nullable, Tuple, etc. It heavily relies on the reflection to do the mapping and parsing.
A good example can be found here. [https://github.com/natemcmaster/CommandLineUtils/blob/master/docs/samples/subcommands/inheritance/Program.cs]
A complete code for the program class:
- Root Command
For our CLI program, we I implement two sub commands for the first version.
login will allow user login with their credentials. The credentials will be saved locally as part of its profile so that user does not need to login again next time.
list-tickets will fetch the tickets from the web service.
We have class
iStradaCmd for the main command, class
LoginCmd for sub command login and class
ListTicketCmd for sub command list-tickets. Also I will have all my command classes inherited from the base class
iStradaCmdBase. That will be helpful since all the command class might want to have some common information like user profile and some common functions like http request/response processes.
Here is how class
iStradaCmd looks like:
The CommandLineUtils library’s attributes are pretty much self-explained.
- Login Command
Login will save the credentials locally in a file, also, it will verify the credentials using the istrada’s api once saved so the user is aware of if they entered the right credential information or not.
the credentials can be passed in through the option -u and -p, or if not presented, the program will ask the credentials input at run time.
Here is how class LoginCmd looks like:
Command login has three options: username, password, and staging. Those three options are mapped to properties on the class through the attributes. In the method
OnExecute, it checks if the username or password is passed in through the command line. If not, it asks the user to input this information since those options are required in order for command to continue to run.
Prompt is a helper from the library that takes the input from console.
Prompt.GetPasswordAsSecureString will display asterisks when user type password from the console.
It writes the user profile to a file under the user’s folder. So next time, the program will load the profile from the file and user does not need to enter this information every time. The password is encrypted when saved in the local file for security.
It then call the API to try to authenticate.
Login support multiple profiles. You can login in as different user and specified a different profile name. Multiple profiles will be saved locally. Later, all other commands can pass the
proflle option to specify which profile to use for that command execution. If option
profile not specified from the command line, it will default to profile ‘
- List Ticket Command
List Ticket command retrieves the tickets from istrada.
end-date options to filter the tickets. If those two options are not presented on the command line, it will prompt the user to enter those options. It then format the url and call the iStrada API to retrieve the tickets. The returned data is in json format. It call
OutputJson to output the data. Currently iStrada API always return data in json format.
OutputJson is a method on the command base class. The base class
iStradaCmdBase will handle how to output the data.
FileNameSuffix is also a property on the base class.
FileNameSuffix will be used to format the filename’s suffix if output the data to a local file.
- Base Command class
All the command classes are derived from the base class
iStradaCmdBase . The base class holds
- some common options that applied for most commands, for example,
- some protected properties that used by inherited command classes, for example,
- some common methods that are useful for all other inherited classes, for example
Here is how the code looks like:
When the class
ListTicketCmd retrieved the tickets, it will calls
OutputJson(tickets, “tickets”, “ticket”);
This is because currently the iStrada API can only return data in JSON.
OutputJson is a method in the base class, the method will check option
output-format, the option default to
json, if the
xml , it will convert the data to XML format using the Json.NET library’s
JsonConvert.DeserializeXNode and then call method
OutputXml. In method
OutputXml, it will check if the command line passed in option
xslt, which points to a local XSL file. If so, will use the XSL file to transform the XML data to another format (xml, html, csv, etc).
The reason we want to support the XSLT transformation is that every customer wants to export the data to their own specific format, even though they might use the same vendor accounting software. By supporting the XSLT transformation, we don’t need to hardcode the data transformation in the code, rather, the transformation file is saved in a separate file that we can modify it without changing the program code.
We also pass a
XSLTExtension object as the XSLT extention to the XSLT transformation. As the .NET only implemented XSLT 1.0. We can use the class
XLSTExtension to implement some features XSLT 1.0 does not support or hard to achieve. The class
XLSTExtension looks like this:
After we transformed the data, now we can output the data. We could output the data to the console as default, or if the command line passed the option
output with a file path, we will save the data to the file.
Decrypt provide the function to encrypt the password of the user profile. It uses the AES (Advanced Encryption Standard) with the key related to the username of the OS. So that, if you copy this file over to another PC or use it under another user, it wont work. This improves the security, but It is not a good enough practice strictly speaking.
iStradaClient is just the wrapper class of iStrada RESTful API.
HttpClient passed in is created by
_httpClientFactory that injected in by the dependency injection from the command class. Here explains why.
- Exit Code
When the program exits, it returns 0 for success or 1 for failure. We can define more codes to indicate the exact reasons of the failure. This code is called exit code. We can define the exit code however we want in the program. But use 0 for success is a broadly adopted convention. If we manually run the program, you can get the exit code by type
echo $? in bash or
echo %errorlevel% in windows command. On Windows/DOS, a pseudo environment variable named
errorlevelstores the exit code.
This exit code some time is important when it is returned to the parent process and the parent process wants to use the exit code to test if the cli command is executed successfully or not. Here is an example in c# on Windows:
With the CLI tool in place, use a script to automate this process becomes a relatively easy task. For example, on windows platform, we can write a PowerShell script to automate this process if customer wants to run this task daily to extract the ticket and save to a local file automatically. And then use the windows scheduler to setup the schedule to run it.
Here is an example
When a lot of users use the CLI tool, one challenge will be how to update the software to the most recent version. Squirrel.Windows is a great .NET solution for the windows desktop software distribution. It handles the installation and update seamlessly.
It is a very simple program. But it does indicate the necessary parts to build a CLI program. As a developer, CLI is more developer friendly than the rich GUI. When we manage our cloud solution on AWS or Azure, or use some framework like .NET Core or Angular 2+, we try to use the command line interface as much as we can to avoid using the more user friendly portal. Those experiences will definitely help us understand how a well-designed CLI program works and some important conventions of the syntax design.