xccov: Diff and Merge Swift(iOS) Code Coverage
Original links:
Refer original links as code formatting and grammar not checked on medium
— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -
Apple has released a new command line tool xccov last year for inspecting the contents of Xcode code coverage reports. With xccov , we can generate Xcode code coverage reports in the human-readable format as well as machine representable format like JSON without using third-party tools. In our previous post, we looked into how to generate code coverage report using xccov tool. In this post, we will explore the new features release with Xcode 10.2.
Code Coverage Path Changed (Xcode10)
We have created a demo repository on Github to explore xccov features. First, let’s build the app and generate the derived data in the Build directory inside the project.
$ xcodebuild -project XCCov-Demo.xcodeproj/ -scheme XCCov-Demo -derivedDataPath Build/ -destination 'platform=iOS Simulator,OS=12.2,name=iPhone Xʀ' -enableCodeCoverage YES clean build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NOAfter running above command, we will get the test report and code coverage reports. This used to generate the code coverage data at path Build/Logs/Test but now reports are generated with the timestamp. e.g
Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-33-02-+0100.xcresult/3_Test/*.xccovreportWe will see code coverage with .xccovreport and .xccovarchive extension.
The real power of xccov comes when it can generate the code coverage reports in the JSON format. As per Apple’s documentation on the man page, its machine representable format. We can hook these JSON results anywhere we like or build another tool on top of this. We can generate JSON reports using the following command.
$ xcrun xccov view --json Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreportWe have just added the flag — json at the end of the command to get the JSON reports.
Diff Code Coverage
As we keep developing apps and adding more source code, Xcode now keeps all the code coverage reports in the different directory with timestamps. In our case, the code coverage reports are stored in the directory
Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07–33–02-+0100.xcresult/3_Test
Now that, we got reports from the specific timestamp, we can diff the reports with a different timestamp to get an idea of what has been changed in terms of code coverage.
As per the documentation, followed the steps but couldn’t generate diff. Following commands tried:
Attempt One:
$ xcrun xccov diff --json Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport, Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportError: Error Domain=XCCovErrorDomain Code=0 "Failed to load coverage report at path Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport," UserInfo={NSLocalizedDescription=Failed to load coverage report at path Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport,, NSUnderlyingError=0x7f93dec23600 {Error Domain=NSCocoaErrorDomain Code=260 "The file “action.xccovreport,” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/Users/shashi/workspace/XCCov-Demo/Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport,, NSUnderlyingError=0x7f93dec23110 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}}
Attempt Two:
$ xcrun xccov diff --json --path-equivalence Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport, Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportError: Must supply the path to two coverage reports to diff
Attempt Three:
$xcrun xccov diff --json from --path-equivalence Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-33-02-+0100.xcresult/3_Test/action.xccovreport,to Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportError: Error Domain=XCCovErrorDomain Code=0 "Failed to load coverage report at path from" UserInfo={NSLocalizedDescription=Failed to load coverage report at path from, NSUnderlyingError=0x7fc2da631430 {Error Domain=NSCocoaErrorDomain Code=260 "The file “from” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/Users/shashi/workspace/XCCov-Demo/from, NSUnderlyingError=0x7fc2da606680 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}}
Attempt Four:
xcrun xccov diff --json --path-equivalence from Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-33-02-+0100.xcresult/3_Test/action.xccovreport,to Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportfish: 'xcrun xccov diff --json --path-…' terminated by signal SIGSEGV (Address boundary error)
As of now, I couldn’t generate the diff from the two code coverage reports. However, I can view both reports when running following command
$ xcrun xccov view --json Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreportIt would be great if there would be an example of how to generate the diff of the source code.
Merge Code Coverage Report
With Xcode 10.2, we can even merge two code coverage reports. We have to provide the path of both code coverage reports and code coverage archive. We also need to specify the path of output. in our case, we ran the following command
$ xcrun xccov merge --outReport ~/Desktop/out.xccovreport --outArchive ~/Desktop/out.acarchive Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovarchive/ Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreport Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovarchive/This command successfully generated out.acarchive and out.xccovreport file on the desktop.
Source Code
The sample code for this post is available on Github XCCov-Demo. You can also see the build data and derived data in the Build directory.
Conclusion
With Xcode.102, you can now create diff files or merge the code coverage from different targets. This would be useful for displaying the code coverage changes on Continuous Integration servers.
Apple has released a new command line tool xccov last year for inspecting the contents of Xcode code coverage reports. With xccov , we can generate Xcode code coverage reports in the human-readable format as well as machine representable format like JSON without using third-party tools. In our previous post, we looked into how to generate code coverage report using xccov tool. In this post, we will explore the new features release with Xcode 10.2.
Code Coverage Path Changed (Xcode10)
We have created a demo repository on Github to explore xccov features. First, let’s build the app and generate the derived data in the Build directory inside the project.
$ xcodebuild -project XCCov-Demo.xcodeproj/ -scheme XCCov-Demo -derivedDataPath Build/ -destination 'platform=iOS Simulator,OS=12.2,name=iPhone Xʀ' -enableCodeCoverage YES clean build test CODE_SIGN_IDENTITY="" CODE_SIGNING_REQUIRED=NOAfter running above command, we will get the test report and code coverage reports. This used to generate the code coverage data at path Build/Logs/Test but now reports are generated with the timestamp. e.g
Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-33-02-+0100.xcresult/3_Test/*.xccovreportWe will see code coverage with .xccovreport and .xccovarchive extension.
The real power of xccov comes when it can generate the code coverage reports in the JSON format. As per Apple’s documentation on the man page, its machine representable format. We can hook these JSON results anywhere we like or build another tool on top of this. We can generate JSON reports using the following comman.
$ xcrun xccov view --json Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreportWe have just added the flag — json at the end of the command to get the JSON reports.
Diff Code Coverage
As we keep developing apps and adding more source code, Xcode now keeps all the code coverage reports in the different directory with timestamps. In our case, the code coverage reports are stored in the directory
Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07–33–02-+0100.xcresult/3_Test
Now that, we got reports from the specific timestamp, we can diff the reports with a different timestamp to get an idea of what has been changed in terms of code coverage.
As per the documentation, followed the steps but couldn’t generate diff. Following commands tried:
Attempt One:
$ xcrun xccov diff --json Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport, Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportError: Error Domain=XCCovErrorDomain Code=0 "Failed to load coverage report at path Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport," UserInfo={NSLocalizedDescription=Failed to load coverage report at path Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport,, NSUnderlyingError=0x7f93dec23600 {Error Domain=NSCocoaErrorDomain Code=260 "The file “action.xccovreport,” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/Users/shashi/workspace/XCCov-Demo/Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport,, NSUnderlyingError=0x7f93dec23110 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}}
Attempt Two:
$ xcrun xccov diff --json --path-equivalence Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport, Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportError: Must supply the path to two coverage reports to diff
Attempt Three:
$xcrun xccov diff --json from --path-equivalence Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-33-02-+0100.xcresult/3_Test/action.xccovreport,to Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportError: Error Domain=XCCovErrorDomain Code=0 "Failed to load coverage report at path from" UserInfo={NSLocalizedDescription=Failed to load coverage report at path from, NSUnderlyingError=0x7fc2da631430 {Error Domain=NSCocoaErrorDomain Code=260 "The file “from” couldn’t be opened because there is no such file." UserInfo={NSFilePath=/Users/shashi/workspace/XCCov-Demo/from, NSUnderlyingError=0x7fc2da606680 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}}}
Attempt Four:
xcrun xccov diff --json --path-equivalence from Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-33-02-+0100.xcresult/3_Test/action.xccovreport,to Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreportfish: 'xcrun xccov diff --json --path-…' terminated by signal SIGSEGV (Address boundary error)
As of now, I couldn’t generate the diff from the two code coverage reports. However, I can view both reports when running following command
$ xcrun xccov view --json Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreportIt would be great if there would be an example of how to generate the diff of the source code.
Merge Code Coverage Report
With Xcode 10.2, we can even merge two code coverage reports. We have to provide the path of both code coverage reports and code coverage archive. We also need to specify the path of output. in our case, we ran following command
$ xcrun xccov merge --outReport ~/Desktop/out.xccovreport --outArchive ~/Desktop/out.acarchive Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovreport Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-40-58-+0100.xcresult/3_Test/action.xccovarchive/ Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovreport Build/Logs/Test/Run-XCCov-Demo-2019.04.03_07-52-58-+0100.xcresult/3_Test/action.xccovarchive/This command successfully generated out.acarchive and out.xccovreport file on the desktop.
Source Code
The sample code for this post is available on Github XCCov-Demo. You can also see the build data and derived data in the Build directory.
Conclusion
With Xcode.102, you can now create diff files or merge the code coverage from different targets. This would be useful for displaying the code coverage changes on Continuous Integration servers.
====================================
Like this post from XCBlog By XCTEQ ? You may also like some of our open source projects on Github or Follow us on Twitter and LinkedIn
