Towards the definitive evaluation framework for cross-platform app development approaches
Christoph Rieger and Tim A. Majchrzak
The following describes how our Journal of Systems and Software (JSS) article Towards the definitive evaluation framework for cross-platform app development approaches has been developed, what its main findings are, and which implications we think it has.
Getting started: what are we doing here?
Most people nowadays own smartphones. Many also own a tablet. And quite likely their TVs, wearables, maybe even their cars are powered by a mobile platform. We call this app-enabled. The typical expectation is that a given app — for example, a social media client — will run on any of these in a comparable quality, with a comparable user experience, and using common user interfaces of the respective platform.
Unfortunately, this means that several apps need to be developed. This is complicated since the two dominating smartphone platforms — Android and iOS — are not compatible regarding app development. Even worse, a developer needs to consider the differences of all the implementations of a platform — particularly one Android implementation is not necessarily fully compatible with another one.
Cross-platform app development approaches promise to solve this problem. There are different approaches towards cross-platform development, ranging on a kind of interval between apps built purely with Web technology to native apps. Selecting the appropriate framework is not easy and reaches beyond technical considerations, particularly if this is not a single-project decision, but a strategic one that you take for many projects, probably enterprise-wide.
We have built a catalogue of criteria, which we propose in our paper. We distinguish development by four perspectives: infrastructure, development, app, and usage perspective. Altogether we distinguish 33 criteria, allowing for a fine-grained assessment of approaches. Moreover, we propose a weighting framework, allowing one assessment to be used for decision making for different applications, e.g. for apps specifically for tablets, wearables, or smart home usage.
Foundations: Towards the first article
One of us, Tim, started to work toward cross-platform apps back in 2011. Then a postdoc at the University in Münster, he engaged in a project with the local chamber of commerce to explore the possibilities of what we named business apps. Only four years after the release of the first iPhone, the smartphone landscape was quite different to what it was today. It was not common that nearly any business that offered a Web site would also offer an app (or typically several apps); many apps for smartphones where limited. Smartphone were not the primary way of accessing the Internet for many people. Data plans were expensive, and the computational capabilities let alone the possibilities of sensors were limited. However, enterprises noticed that there was a demand to provide customers with apps for specific pieces of functionality that would reasonably be used from a smartphone.
The project had a quite curious setup. We worked with regional industry, doing interviews and talking to developers, but the subject of study was really global. Unsurprisingly, we could publish a paper on the project reporting results that were perceived noteworthy [MaHe13]. In the meantime, we had engaged in various activities on the topic, which we found to be underrepresented in research. Our approach of tackling it followed a path typical for new research topics: we started by working exemplarily on case studies [ZiMa12], covered broader topics [HMRW13], and eventually set out to contribute to the theory [HKM15].
In parallel, we wanted to work practically on the problem of cross-platform app development, but with scientific rigour in mind. Cross-platform app development can essentially follow two approaches. Either they use a runtime environment, or they are generative. Generative approaches yield truly native apps but use source code for one app only. Runtime based approaches provide a layer between app and mobile platform; such approaches are available with varying sophistication, from such that are more or less Webapps to such that use the native GUI elements through a self-contained runtime. Approaches developed and favoured by industry are based on runtime environments because of the availability of programmers with related experience (e.g. JavaScript). As computer scientists (or rather information systems people who like computer science methods), we favoured the generative approaches, which would in theory yield “cleaner” results and offer many benefits regarding semantic analysis and performance.
A few transpilers were available, but their performance was not satisfying. A transpiler takes code written in one language for one platform and transforms it to code written in another language for another platform. Due to platform and language idiosyncrasies, particularly when it comes to the GUI, this is very hard. Therefore, we favoured a model-based approach: instead of instruction how an app should work, we set out to ask developers to merely specify what an app should be capable off in a concise and platform-neutral representation. Work on our prototype MD2 proved to be very fruitful, although it remained an academic prototype [HMK13]. Learning that cross-platform app development ought to follow software engineering best practices, be theoretically sound, yet industrially producible made us think. Platform fragmentation, device heterogeneity and rapid release cycles made (and make) it hard for developers. Therefore, through a Bachelor thesis we had a student analyse these early cross platform app development frameworks. The results seemed so promising that we invited him to work with us in turning their results into a scientific framework and revising it further. The work proved even more relevant than we had anticipated. The article from 2013 [HHM13] has been cited more than 330 times.
Work: Getting the article done
After serving as deputy professor in Cologne, Tim started as tenured associate professor at the University of Agder in 2015. He set out to enable a second research tier, becoming a member of CIEM (Centre for Integrated Emergency Management) — societal support had always been a theme Tim was interested in. However, collaboration continued with the group in Münster, where Christoph started as PhD student in 2015. One stream of research focused on extending previous cross-platform approaches towards novel mobile devices such as smartwatches and voice-based smart personal assistants. Therefore, a graphical notation called MAML [Ri17] was built on top of MD2 to demonstrate the transferability of abstracted functionality to heterogeneous devices.
Motivated by the success of the first evaluation paper from 2013, Christoph and Tim decided to update the original paper. Despite its impact, it offered room for improvement. First, it had become dated due to the unprecedented progress in the field with platforms and frameworks frequently occurring and disappearing. Moreover, with Progressive Web Apps (PWAs), a new contender recently entered the scene and recent frameworks needed to be addressed [BMG18]. Second, it did not cater for the different application areas of mobile platforms — now not limited to smartphones anymore but spanning for example entertainment electronics, vehicles, and wearables [RiKu18]. Third, observation of the application of the original research indicated that with the evolution of the field in practice, the theoretic base could be refined and the framework slightly tweaked to flexibly accommodate the unique requirements of cross-platform projects.
The first step to achieving this goal was to offer a weighting framework on top of the revised original framework [RiMa16]. Since we got encouraging feedback when presenting our ideas at a conference, we decided that our work finally was ripe to contribute to theory. The idea for the JSS paper was born in September 2016. A journal paper, particularly one that builds on a broad base of literature and seeks to theorize, takes some time to write. After more than two years and many internal revisions, our article was handed in on 20 November 2018. Thanks to the quick JSS review process, we soon received comprehensive feedback. Much to our relief, the time we had taken was honoured and most comments tried to push us even further, e.g., for thoroughly linking our extensive set of criteria to the scientific body of knowledge. On 25 February 2019 we uploaded our revision and received the acceptance notification on 1 April 2019 — it was not an April fool’s joke!
Conclusion: Where we hope this to be going
To give a better idea of our framework, the following table shows an exemplary set of weights for different device categories as described in detail in the paper. Consider it a snapshot and really exemplary to illustrate the evaluation of one of the various. The evaluated platforms have since then been developed further, some more, some less. Moreover, categories of novel app-enabled devices (like vehicles) are quickly evolving and would need to be updated. However, the weights always need to be decided individually anyway, since the framework is meant to be adjusted to the needs and preconditions of developers, the companies they work for, their customers, and possible end-customers.
Despite being published in JSS, our article has not yet been cited as much as the first evaluation article from 2013 had. We hope that it has been noticed in practice for its comprehensiveness, though. The need of developing interoperable functionality in a cross-platform fashion keeps being there, although modern Web-based development frameworks and Progressive Web Apps (PWAs) have likely shifted the focus toward Web technologies in many cases.
Research on the underlying topics is ongoing. However, it has become much more specific. For example, the performance of cross-platform app development technologies is now studied in detail which is reflected in the latest evolution of frameworks towards hybrids of runtime and generative approaches. In addition, research institute Gartner predicts that “multiexperience” apps with immersive interaction modes such as chat, voice, wearable, and augmented reality experiences will become much more common in the near future [Ga19]. Further software engineering research is mandated to boost the efficiency of app development and cope with the plethora of heterogeneous devices.
Our further journey with this topic is yet undecided. Whereas Christoph closely follows recent developments in the field of mobile app development, he has completed his PhD and left academia to work on technology-related topics in the private sector. Tim stayed and still engages in topics in app development. From our observations, the convergence of technologies might become the next roadblock on the path towards improving cross-platform app development — both challenging and interesting at the same time. Mobile computing, the Internet-of-Things and cyber-physical systems (CPS) share an increasing number of constituting elements. It may be that they soon also share platforms and means of development much stronger than they do now.
Interested researches and practitioners will find good venues for their work on the intersection of Web technology, mobile computing, IoT, and recently also cyber-physical systems in the HICSS minitrack on Software Development for Mobile Devices, the Internet-of-Things, and Cyber-Physical Systems as well as on the SAC track on Web technologies, which Tim both co-chairs.
References
[BMG18] Biørn-Hansen, A., Majchrzak, T. A., and Grønli, T.-M. (2018): Progressive Web Apps for the Unified Development of Mobile Applications. In: Majchrzak, T. A., Traverso, P., Krempels, K.-H., and Monfort, V., eds.: Revised Selected Papers WEBIST 2017, Lecture Notes in Business Information Processing (LNBIP), 322, 64–86, Springer
[Ga19] Gartner, Inc. (2019). Gartner Says the Future of App Development Is Multiexperience. https://www.gartner.com/en/newsroom/press-releases/2019-04-04-gartner-says-the-future-of-app-development-is-multiex
[HHM13] Heitkötter, H., Hanschke, S., and Majchrzak, T. A. (2013): Evaluating Cross-Platform Development Approaches for Mobile Applications. In: Cordeiro, J. and Krempels, K.-H., eds.: Revised Selected Papers WEBIST 2012, Lecture Notes in Business Information Processing (LNBIP), 140, 120–138, Springer
[HMK13] Heitkötter, H., Majchrzak, T. A., and Kuchen, H. (2013): Cross-Platform Model-Driven Development of Mobile Applications with MD² In: Proc. of the 28th ACM Symp. on Applied Computing (SAC), 526–533, ACM
[HKM15] Heitkötter, H., Kuchen, H., and Majchrzak, T. A. (2015): Extending a Model-Driven Cross-Platform Development Approach for Business Apps. Science of Computer Programming (SCP), 97(1): 31–36
[HMRW13] Heitkötter, H., Majchrzak, T. A., Ruland, B., and Weber, T. (2013): Evaluating Frameworks for Creating Mobile Web Apps. In: Proc. 9th International Conference on Web Information Systems and Technologies (WEBIST), 209–221, SciTePress
[MaHe13] Majchrzak, T. A. and Heitkötter, H. (2013): Development of Mobile Applications in Regional Companies: Status Quo and Best Practices. In: Proc. 9th International Conference on Web Information Systems and Technologies (WEBIST), 335–346, SciTePress
[Ri17] Rieger, C. (2017) Business apps with MAML: a model-driven approach to process-oriented mobile app development. In: SAC: 1599–1606
[RiKu18] Rieger, C. and Kuchen, H. (2018) Towards Model-Driven Business Apps for Wearables. MobiWIS 2018: 3–17
[RiMa16] Rieger, C. and Majchrzak, T. A. (2016): Weighted Evaluation Framework for Cross-Platform App Development Approaches. In Stanislaw Wrycza, ed.: Proc. of the 9th SIGSAND/PLAIS EuroSymposium on Systems Analysis and Design Information Systems, Lecture Notes in Business Information Processing (LNBIP), 18–39, Springer
[ZiMa12] Zibula, A. and Majchrzak, T. A. (2012): Developing a Cross-Platform Mobile Smart Meter Application Using HTML5, jQuery Mobile, and PhoneGap. In: Proc. 8th International Conference on Web Information Systems and Technologies (WEBIST), 13–23, SciTePress
Author Biographies
Christoph Rieger received BSc and MSc degrees in Information Systems from the University of Münster, Germany. By contributing to MD2 framework during his Master’s thesis, he combined his interests on mobile devices and software engineering which became focal areas of his later work as a research assistant in Münster. In particular, model-driven software development and domain-specific languages for several application areas are central topics of his research which was published in renowned conferences and journals including Journal of Systems and Software (JSS) and Empirical Software Engineering (EMSE). After receiving his PhD in economics (Dr. rer. pol.), he now pursues his interests in technology-related topics as a consultant.
Tim A. Majchrzak is professor in Information Systems at the University of Agder (UiA) in Kristiansand, Norway. He also is a member of the Centre for Integrated Emergency Management (CIEM) at UiA. Tim received BSc and MSc degrees in Information Systems and a PhD in economics (Dr. rer. pol.) from the University of Münster, Germany. His research comprises both technical and organizational aspects of Software Engineering, typically in the context of Mobile Computing. He has also published work on diverse interdisciplinary Information Systems topics, most notably targeting Crisis Prevention and Management. Tim’s research projects typically have an interface to industry and society. He is a senior member of the IEEE and the IEEE Computer Society, and a member of the Gesellschaft für Informatik e.V.
