Contributing to your preferred open source project, starring Flutter — part 2

How to configure your tools to edit native code, or flutter SDK.

• Part 1: https://medium.com/@fmatosqg/ill-show-you-how-you-can-add-a-completely-new-feature-to-flutter-framework-in-3-short-steps-9783db763ecc, covers the less technical parts
• Part 2 : this one, covers how to configure and deal with IDE and environments
• Part 3: https://medium.com/@fmatosqg/contributing-to-flutter-part-3-4b96d924d8ba, covers the tale of a flutter issue

In this post I’ll cover

• how to modify a package’s source code in either Dart, or Native, and use it in your project
• how to modify Flutter SDK source code and use it in your project

There will be a resonable amount of content to cover, but hopefully we’ll see more things than you need.

I’m assuming that if you need to use a particular flutter/android/iOS feature you both know the language and the framework you’re writing on. What I’ll show instead is how to use the tools you already know, to write that code and do all sorts of TDD and library/app development. I also assume you’re familiar with git and github. But even if you don’t you can still get some understanding of how the cogs move each other.

Editing Android or iPhone native source code

Disclaimer

• VS Code is not really an alternative for this section. To the best of my knowledge it doesn’t have support for Android or iOS native projects. But if it’s your thing use it at your own risk. BTW Jetbrains has an IDE for iOS development called AppCode. Trying to use Android Studio or Intellij to edit iOS native projects as likely to result in little more than frustration.
• The only operating system that can compile files for iOS is Apple’s. You can still edit source code, but the compiler and linker necessary to build the files accepted by the app store only run on OS X.

Editing your own apps’ native code

The following steps apply if you want to modify the native source code from either an app or a package.

• Open flutter project on Android Studio (AS)
• Go to menu tools, select flutter
• Choose ‘Open Android module on Android Studio’ (this does the same as opening an existing project in AS, pointing to <your flutter project root folder>/android)
• Or you can choose ‘open ios module on XCode’ (this is the equivalent of opening XCode and then opening project <your flutter project root folder>/ios/Runner.xcworkspace)

Check out, we even have an option for writing feedback right here. It will immediately create a new issue in the AS plugin repository (not the SDK one) https://github.com/flutter/flutter-intellij/issues/new

Android Studio and XCode users should be familiar with what to expect next: you can benefit from proper syntax highlight, code completion and debug breakpoints on native code.

You can also run the flutter project from here. Gradle has a reference for a flutter plugin, who will compile dart files followed by the usual build process. XCode still uses CocoaPods to reference external code and hook into Dart compiler. There’s no support for hot reload, although the usual native environment works exactly as expected.

Is it worth using the native IDE?

You can also access everything else those tools offer, such as: android/app/build.gradle, where you have native build setup, dependencies configuration and app signing. Or android/app/src/main/AndroidManifest.xml, that hold permissions, screen orientation and more.

Android studio from flutter app, showing included package video_player as a submodule

There’s other reasons to use such tools. If you want to configure Android’s status bar you can edit android/app/src/main/res/styles.xml. For example to make it go away you’d add <item name=”android:windowFullscreen”>true</item> to LaunchTheme. Also XCode is notorious for having nice interfaces for editing files with proprietary formats (varying from mildly hard to impossible to either read or solve merge conflicts). Or the plist file for iOS, which stores a great deal of information of all sorts. Most of those files will have linters and compilers running before you think about running your app, suggesting changes and providing autocomplete so the effort to type the correct thing is greatly reduced. And preventing you from running the app until you fix syntax and compiler errors, which is actually a big time saver. I still wish

Eventually you’ll want to configure a package that needs editing some of those files. For example when you add firebase or facebook to your project, either through pubspec or directly on native configuration. What’s quite interesting , when you want to add a file to an iOS project you absolutely need to do it by opening XCode. Then you drag and drop the file on it, and it gets added to project.pbxproj that registers all files in your project. Adding the firebase json file is one such example.

XCode showing sibling native project, displaying included flutter package video_player

Understanding the build system for packages

Once you open the native IDE you also have access to all the source code and related files from all the included flutter packages. They are not subfolders, but from the way the build system references them they will pop-up as if they were, as submodules/pods in your project. I got quite a surprise when I discovered this: packages don’t include code in binary form, and it’s your project that compiles and processes all of that — dart, and java, and swift, as well as any assets. I guess that if you have a big project with lots of dependencies the full compile time will get quite big quite fast.

To better understand how flutter builds apps I’d encourage you to quickly explore into the code for one of those packages. If you make changes there and relaunch your app you’ll be able to immediately see changes. It’s an interesting learning tool, but I strongly discourage you from changing package code this way. On one side you may not know how to undo the changes, because git won’t be tracking those folders — they live on ~/.pub-cache/hosted/pub.dartlang.org/. On the other hand you’ll see those changes applied to all projects that import those versions of such packages. Do it at your own risk, but if you want to do it for legitimate purposes know there’s a right way to fiddle with packages’ code and this is not it.

That’s how:

• clone such repository somewhere in your disk
• change your pubspec.yaml to something like this:

from something like this:

dependencies:
  flutter_staggered_grid_view: 0.1.2

to

dependencies:
  flutter_staggered_grid_view: “0.1.2”
. . .
dependency_overrides:
  flutter_staggered_grid_view:
    path: /path/to/flutter_staggered_grid_view

Or even commit and upload your changes, that’s an effective way of testing with your co-workers or making a company-wide fork:

dependencies:
  flutter_staggered_grid_view: “0.1.2”
. . .
dependency_overrides:
  flutter_staggered_grid_view:
    git:
      url: https://github.com/letsar/flutter_staggered_grid_view.git
      ref: 938575 <replace with branches, tags, etc>
      path: <path to folder containing pubspec_file, usually ommited>

For more details on syntax for dependencies see https://www.dartlang.org/tools/pub/dependencies#path-packages

If you did modify those files and want to revert everything back to normal, a pub cache -repair will re-download all packages you have on your disk. Or you can also delete . ~/.pub/cache followed by flutter packages get, so you dump all packages and only download the ones you’ll be using immediately.

(thx @zoechi and @Mahi-K for pointing in the right directions)

Once you want to run your recently typed changes forget about hot reload. This code is not interpreted, so you will have to stop the app and launch it again. Though Android allows you to do hot swap in traditional Android development I haven’t tried it yet — search for Instant Run for more info. And the full restart is actually not that full, so you really will need to see the app go away and come back. I find that confusing, so I added some notes here https://github.com/flutter/flutter/issues/16493.

Opening the Android project of a package, starting from your flutter app project

Why is this all important?

There’s a whole world of APIS and documentation built over a decade, it’s just impossible to access all of it from flutter. And if Fuchsia ( the new experimental mobile OS ) comes out virtually none of the existing packages will work. As the platform is relatively new there’s a lot of low hanging fruits, and I hope my contribution will help cover that gap. Some that I noticed are

• the connectivity plugin doesn’t have an api for showing the wifi or mobile IP address. This requires a little bit of Dart and a handfull of java/kotlin and swift code.
• there’s no way to dynamically hide the status bar. I couldn’t find a package closely related to that, so that would involve creating one from scratch. SystemChrome.setEnabledSystemUIOverlays allows you to move it around (for fuchsia support?), so it may be a good place to add it.
• the animations’ interpolators still don’t include a Curve that implements springs. This can be done writing Dart code alone, although peeking into the existing java implementation can save some time. And you can brag with your React Native colleagues that they had to do it in java because of bridge speed issues.
• a plugin to read phone number (without relying on firebase)
• add github authentication for the firebase_auth package — @osorito will be happy :). Get in touch with him if you plan doing it.
• a plugin to read/write images to the gallery, which could also provide path to other common folders in android (the concept is new on iOS, not sure how it works)

The last one could be a bit more complicated: a) add permission to read external memory on manifest b) get permission to read on external memory c) find the folder where gallery gets stored (with path_provider) d) download file and write there e) call the appropriate Android method to scan for images (if you skip last step you’d have to reboot phone).

Some useful resources

• How to use flutter channels, so you can have Dart code interoperability with native https://flutter.io/platform-channels/
• How to create packages from scratch https://flutter.io/developing-packages/
• React Native implementation of spring animation https://github.com/facebook/react-native/blob/master/ReactAndroid/src/main/java/com/facebook/react/animated/SpringAnimation.java
• The official gitter channel, where there’s a lot of devs who helped me get where I am: https://gitter.im/flutter/flutter (yes I’m calling gitter a resource, not the people)

And non-tech resources

• Flutter guide to contributing to SDK source code, from where I got a lot of information for this blog: https://github.com/flutter/flutter/blob/master/CONTRIBUTING.md
• Google advice on code reviews: https://testing.googleblog.com/2017/06/code-health-too-many-comments-on-your.html

Flutter SDK development

This covers changes to the part of flutter written in Dart that all developers use when building their apps and libraries.

There are 3 important branches you must know about. beta is the stablest version we have, and it gets update quite… not often. Then there’s the dev branch which has less stable code, and finally master where every new piece of code goes first. These 3 branches are also called channels, and so far they’re the only official channels we have.

Unfortunately they also coined the term channel to designate how to communicate between dart and native code. That’s a completely unrelated topic, but I’d rather mention it to avoid confusing both.

Now in beta 2 there are 2 ways to install flutter. You can download a big fat zip or you can clone from https://github.com/flutter/flutter. Either way is fine, but I recommend you make a second copy of whatever you have. So my suggestion is that before making any planned changes you keep your main copy on beta to keep writing stable apps, and your working copy on master so you can put it on steroids. The reason is that when you find odd behaviour you can quickly recompile your app with the beta and rule out any quirks.

Using alternative installation for flutter

Simply copy the folder in your preferred way, there’s nothing mystical about a flutter installation. Let’s say your main copy will stay on ~/bin/flutter and your dev copy will be on ~/code/projects/flutter.

Now let’s update our tools to point to the new folder. One way is to put this in your .bashrc

alias flutter_experimental=’export PATH=~/code/projects/flutter:$PATH’

and every time you want to use it in a fresh terminal do

flutter_experimental

And on either Android Studio or Intellij go to preferences / Languages and Frameworks / Flutter. On the field for “Flutter SDK” path put ~/code/projects/flutter. Just remember to do that for every flutter app or plugin you want to compile, it won’t be carried over to all projects at once.

Yes, it does mean you don’t have to wait for google to approve your changes so your team benefits from your changes! Life is good, and is about to get better!

Now change to the master channel, followed by getting all the latest changes. You can do that with:

flutter channel master
flutter update

Instead you can also use the usual git commands and get the same results.

cd <wherever your flutter is installed>
git checkout master
git pull origin master

And you can use these steps to have at hand a working copy for each channel, to quickly switch whenever you need.

At this point you probably want to fork on github. Login and hit the fork button on top right.

I always use command line, but GUI tools also have a way of adding a second remote repository to a folder. Mine was added like this:

git remote add upstream git@github.com:fmatosqg/flutter.git

where fmatosqg is my user.

This way you can still get changes by doing git pull origin master while pushing your changes to git push upstream my_feature.

Changing sample code

Just as an exercise, let’s add a log to the file that handles the manifest.

Open Android Studio or VS Code. Then open project, on folder ~/code/projects/flutter/packages/flutter_tools. Check that Dart is configured and after that open file lib/src/flutter_manifest.dart.

An alternative way on AS is to quickly tap shift (it only works with the shift to the left of the keyboard). Type FlutterManifest and pick flutter_manifest.dart from the list.

Find method createFromPath and on its first line add print(‘Hello World’);. That’s it for editing.

Tips: to quickly see and access the classes and methods in a file open the flutter outline panel — don’t forget to turn off the filter by views option, which is the funnel on the top right. You can also use the structure panel, accessible with Cmd+7

Now open a second Android Studio instance (or VS) and open a project with a flutter app.

As explained in previous section, change your configuration — either command line or AS or VS — to point to your modified copy of flutter.

Launch the app and see your hello world message show up.

Now let’s change it again, to print(“Hello Manifest”);

Stop the app, relaunch and… nothing changed. That’s because flutter optimizes and caches the result of its last build. To trigger it again:

• Delete bin/cache/flutter_tools.snapshot, it’s on ~/code/projects/flutter/
• flutter clean to delete app’s build folder

Stop the app one more time and start it again. You’ll notice it takes a little longer, it crunches that change you just made, and finally prints it on screen.

Running tests

You can also run a particular unit test in AS by opening the test file, say, packages/flutter_tools/test/flutter_manifest_test.dart. On the top of that file there’s a main method. Right click it on the editor window and choose either Run or Debug. You can also run individual groups of tests, doing the same with either the group method or the individual unit tests.

Now you’re ready to do your magic. When you’re finished and before sending a pull request to github remember to check the dart analyser from inside AS. And that all unit tests are successful.

flutter packages pub run test  | tee test.log

I like to do it from AS’s own terminal:

Epilogue

Flutter and React Native are based on the principle of Learn Once Write Everywhere. It doesn’t mean that you can do it without access to OS X.

It also means you can write apps that do exactly the same thing on all platforms it supports, but not that you should do it. Users expectations are very deep into their behaviours, and breaking those patterns must be a conscious choice. There are big differences, such as the Material vs Flat design and the ways to deal with background operations. There are also subtle things, such as the friction coefficient of scrolling views and how you should ask the user that the app needs access to GPS. Not to mention all the rules Apple want you to follow before it approves your app in the store.

Also some gotchas, as expected flutter draws every pixel on your app. That’s true, but technically the status bar doesn’t belong to the app, instead we kindly ask the operating system to perform a couple of operations on it on our behalf. In the same sense, all sockets opened by the Dart code will obey to rules set up in the app, who will ask the operating system for it. If you don’t include internet permissions in the manifest (or a flutter package does it for you), or if the SSL certificate from the server is not accepted by the OS, there’s no possible way to get a single bit from the cloud. The same goes for asking for permissions before sending push notifications. Unless dealt according to what each ecosystem needs it won’t just work.

You can get away with a lot by only learning flutter and dart. When you cross from Dart to native you have to deal with platform specific documentations. And when you don’t you’ll still be occasionally tapp into that. You still need XCode to build for iPhone, and understand its ecosystem properly when setting push notifications, configuring provisioning profiles and deploying to the store. Android builds still rely by and large on gradle, Android’s SDK, java and also on NDK, and it may be useful to understand them when something breaks — at least so you know whether it’s a package import problem or it’s coming from something you wrote in your app.

These examples are just to illustrate that flutter is “simply” a layer on top of the existing Android/iPhone tools and source code and operating system. It’s quite an impressive project, and it implements the UI code from a very low level, arguably even more native than Android native, but almost everything else is built on top of the same tools and procedures that native apps use normally.

Happy coding!