CodeX
Published in

CodeX

CODEX

Building the Nativase Web App — Motivation & Methodology

20% of all requests to one of Wikimedia’s data centers in early February 2021 were for this stock image of the New York aster (symphyotrichum novi-belgii), a flowering plant endemic to eastern Canada and northeastern America. The culprit: an India-based mobile app that requested the image, but did not display it. Source: Wikimedia Commons

And yet before we take our implements to unfamiliar territory we must work to ascertain its changing weather and winds’ moods to learn the ways and habits of that locality — what’s bound to flourish there, and what to fail.
For here you’ll find a crop of grain, and there grapes growing in thick clusters, and over yonder young trees thriving and grasses coming into green all on their own.

- Book 2 of Virgil’s Georgics (translated by Peter Fallon)

App: https://nativase.shinyapps.io/Nativase/
Github:
https://github.com/liamtrotzuk/nativase

After 6 months of building a web app, it felt like a good time to pause and reflect on what I’ve learned — as someone who undertook a nice project with no formal software development training or interface design background. In summary: web app creation is a very fun process, and I would recommend it to almost anyone. Pick the right datasets and the right presentation, and the actual construction of the app will basically take care of itself, because you’ll be so engaged with your project that any hurdle will be surmountable with enough Googling and pondering of the fastest way you can muddle through whatever problem you’re facing. It’s a great way to spend some free time.

The following is a short summary of my initial motivation for building the app, and a simple methodology for web app development that I refined along the way.

Motivation

Like many folks in 2020, I spent much of the year indoors. And like many folks, I began to reflect on ways to improve the space where I had suddenly begun to spend a disproportionate chunk of my days. A new carpet might be nice; maybe a fresh coat of paint on the walls; perhaps a fun work of art to hang up. But as I considered various ways to brighten my space, 1 particular option began to dominate my imagination: furnishings with life. Plants — green, growing things to fill my living space. Floral fractals to ease the eyes. Steps toward solarpunk.

I’m not the only one. The pandemic life of 2020 and accompanying social distancing across much of America led to an unprecedented boom in gardening and plant care. Garden centers and seed suppliers experienced a steep rise in demand. Concerned by the long lines and empty shelves at grocery stores in March/April, would-be Thoreaus planted pandemic ‘victory gardens’ to grow at least a little food for themselves, thus uncoupling even slightly from the just-in-time supply chains of modern supermarkets. The Wall Street Journal published an article on the red-hot trade in rare plants.

Like many of the pandemic’s newly minted plant parents, I had never been a huge ‘plant guy’ in my pre-2020 days. I always considered plants to be a needless chore to buy (stems/fronds/leaves are delicate), to plant (soil is messy), and to tend (houseplants need to be carefully watered and sunned). But growing things has been a human birthright for the past 12000 years, and there are certain advantages to approaching a hobby a bit later in life than others, with a completely fresh and naive set of eyes. For me, I began to carefully weigh the costs and benefits of different types of gardening, planning from the ground up. Would I grow plants inside in pots or outside in window boxes? Would I buy mature plants or grow from seed? I considered growing herbs for use in the kitchen —basil, thyme, bay leaf. I toyed with the idea of growing plants favored in medieval times for use in powerful potions —sage, betony, deadly nightshade. I thought about buying or building an automated hydroponics system to grow cherry tomatoes or hot peppers.

Then I started thinking about Ulmus americana, the American elm.

The American elm is an elm tree native to the Americas, specifically the eastern coast of North America from Florida to Newfoundland. They are resilient trees that can live for hundreds of years. They are also quite beautiful: a grove of mature American elms shades Central Park’s Mall, which is a lovely promenade just south of Bethesda Fountain and east of Sheep’s Meadow where one can have a pleasant stroll in the shadows of those grand old trees. That part of Central Park is encircled by the tall skyscrapers of midtown Manhattan, which leads to a unique effect on those walking: surrounded by majestic trees, the walker may forget — even just for a moment — that they are in the core of an enormous city that is home to 20 million humans in its greater metropolitan area. But then through the organic wooden limbs of the American elms that surround them, the walker will inevitably glimpse again the tall monolithic shapes of skyscrapers that ring the park, cutting vertically into the sky all around. It’s a nice contrast.

Unfortunately, Ulmus americana is also an endangered species, now barely surviving in the native land where it once grew in vast forests from the Atlantic to the Mississippi. The primary culprit: Dutch elm disease, a fungus spread by elm bark beetles with its origins in Eurasia, against which American elms have no natural defenses. Dutch elm disease spreads from elm to elm, burning through forests of the native trees with alarming speed. By the end of the 20th century, once-sprawling forests of American elms had been reduced to a mere handful, mainly concentrated in urban groves inside American and Canadian cities where the Dutch elm disease running rampant in forests could not spread so easily. One such grove — one of the largest remaining groves of American elms in the world — shadows the Mall in Central Park, New York, NY.

A grove of American elms at the Mall in Central Park, NY. Source: Wikimedia Commons

One only has to think about that fact for more than one second to realize how absurd it is. One of the largest remaining groves of American elms lies not in some sprawling untouched massive swathe of wild forest in the Appalachians, Michigan’s Upper Peninsula, or Nova Scotia, but in the very center of the densely populated urban island at the heart of the largest metropolitan area in the United States of America.

And just like the American elm, many other native American plants — and the native American herbivores that depend upon the plants for food, as well as the native American carnivores that depend on the herbivores for food — are under serious threat.

Wild horses (Equus ferus caballus, from Europe) overgraze vast Western ranges. Kudzu vines (Pueraria, from Asia) strangle Eastern forests. Lionfish (Pterois volitans, from the Indo-Pacific) snarl Caribbean reefs. Most recently, the arrival of Japanese murder hornets in the Pacific Northwest in the spring of 2020 made ripples well beyond the ecological sector, as folks worried about the devastating impact these invasive hornets are known to have upon beehives — which is fair, except that common honeybees are non-native themselves. Most honey in North America is produced by Apis mellifera, the European honeybee (though there are several species of endemic American bees that also produce honey — they remain underutilized by all except for a few beekeepers). While European honeybees face their fair share of threats — including murder hornets and pesticide-induced colony collapse disorder — native bee species face a more dire threat altogether. A report from researchers at the National University of Comahue in Argentina highlighted that fully 1/4 of all documented bee species have not been seen in recent years, suggesting extremely sharp declines in the populations of these species, if not outright extinction in some cases.

As non-native organisms spread, so do endemic organisms like the American elm retreat. Douglas Tallamy, a professor of ecology at the University of Delaware and well-known figure in ecological literature who advocates for the planting of native species, writes in his New York Times best-seller Nature’s Best Hope: “Populations of birds, plants, and animals — such as Kirtland’s and golden-cheeked warblers, Florida panther, marbled salamander, spruce grouse, Winkler’s blanketflower, Karner blue butterfly, lynx, bobolink, lake sturgeon, whooping crane, California condor, regal fritillary, rusty blackbird, gopher tortoise, roseate spoonbill, American ginseng, indigo snake, whip-poor-will, Catesby’s lily, Florida scrub jay, dozens of fish and mussel species, and many, many more — became too few in number to perform their vital ecological roles or to withstand normal environmental challenges.” The majority of ecologists believe that this decline in native fauna is directly linked to the decline in native flora.

On a more personal level, New York City — my hometown — was the staging ground for just such a collapse in biodiversity, and on a local level too. Mark Kurlansky, in his nonfiction history work The Big Oyster, describes this jewel of Lenape territory as it existed when Europeans first journeyed to these waters: a teeming, productive ecosystem centered upon the Upper and Lower bays of New York, with rivers so full of fish that they could be pulled from the water by hand, flocks of wild turkeys that shut out the sunshine, and an abundance of predators terrestrial, marine, and avian — including whales, wolves, and eagles — to feed upon the native cornucopia. At the center of it all — facilitating the life and in turn being facilitated by the life — were the massive oyster beds of the area, which fed by filtering the rich water and thereby permitted the continued health of the other inhabitants of the bay. Kurlansky writes: “According to the estimates of some biologists, New York Harbor contained fully half the world’s oysters.”

That’s all gone now, of course. While humpbacks occasionally make headlines by journeying up the Hudson River, and the Billion Oyster project works to restore at least a fraction of the giant oyster populations that used to exist here (among many other promising environmental initiatives in the area), these days we generally think of NYC as a city 1st and an ecosystem 2nd.

Source: Wikimedia Commons

Has that begun to change, in NYC and in the US as a whole? If these ecological troubles had all taken place decades or centuries ago, we might assume, Americans would’ve since learned to do better and accordingly adjusted their ways to eat sustainably, reduce pollution, and — of course — plant native, with the hopes that endemic species should start to rebound. But Tallamy, who has published several books advocating for a return to planting native species rather than foreign ones, laments the ongoing problem: despite all the evidence, American people keep planting and proliferating non-natives. And the potential damages from this continued practice are piling up. As mentioned, invasive species are a concern, and Tallamy mentions that we often plant, water, and fertilize these potential invaders in our own gardens: “about 85% of invasive woody plant species in the United States are escapees from our gardens (Kaufman and Kaufman 2007).” Another issue: non-native plants are often poorly adapted to their surroundings and require large amounts of water and fertilizer to thrive, further taxing local ecosystems. And most subtle of all (but possibly also most harmful) is the opportunity cost of non-natives inhabiting the very space that natives could use. Every acre devoted to flat, beautiful, but ultimately sterile seed lawn is another acre that cannot be used for a rich field of native grasses. Every city block planted with ginkgos rather than native trees is another urban canopy in which caterpillars will not grow and birds will not feed.

To be absolutely fair: obviously non-native plants can have enormous value when cultivated outside of their places of origin, and to pretend otherwise is absurd. Take agriculture. Pest-resistant varieties of African rice have nourished the coastal Americas for hundreds of years. Feng Xiaoyan, a Chinese folk performer known as ‘Sister Potato’, sings about the wonders of the (originally Peruvian) potato in a Beijing-sponsored food campaign to improve crop yields in the PRC. And where would contemporary Italian cuisine be without a certain red vegetable, belonging to the nightshade family and originating in Central America — the tomato? More importantly, those examples lie only in the agricultural world. Exchange of plant matter across borders, deserts, and oceans has led to enormous advances in other industries as diverse as textiles, construction, and pharmaceuticals — to name only a few.

And endemic plants alone are obviously not a silver bullet for preserving biodiversity. Maize is a plant endemic to the Americas, and vast monocultures of the stuff blanket entire states in the Midwest, a relentless optimization of calorie yields that might well be inferior in the long run to a sustainable, diverse polyculture of the sort once cultivated by indigenous peoples of the Americas — maybe even a polyculture that included some non-native plants.

But those American elms, on the brink of extinction but somehow preserved there in Central Park — they really make you think. Climate change, pollution, and the despoilment of the natural world that surrounds us will require wholesale realignments in many of our lives, especially in the industrialized Western nations that emitted most of the emissions of the past few centuries and therefore must shoulder most of the responsibility for fixing the problem. There is no 1-step solution — there are more like 1000 steps, steps that must be taken by corporations and individuals alike. But each 1 of those 1000 steps must be taken sooner or later, and there were those native elms doing just fine in the middle of the city, and if I was going to plant (and I’d already decided that I was) — shouldn’t I just go ahead and plant native?

So I did. Butterfly milkweed, New York fern, bottle gentian, cattail sedge, and old man’s whiskers (AKA prairie smoke) to start, with more to come this year and next. It’s an exciting journey that I’ve only just started. I haven’t completely ruled out non-native plants, as they do have important roles even when growing in locations far from their sites of origin, and with careful management should continue to yield great benefits for the continued endurance and success of humans, animals, plants, and other forms of life on Earth. But for the foreseeable future, I’ll be planting native—those species can use the help, and probably will for a while to come.

Still — it didn’t feel like quite enough. I realized pretty quick that my apartment window plants will have limited ecological effect. Some insects may pollinate their flowers, some birds may peck at their berries, and that’s all to the better. But I have no land to leverage —no lawn to turn to native grasses, no backyard to seed with endemic wildflowers, no rolling fields of estate to plant with American elm. There are other, public spaces to use in the city— fire escapes, rooftops, backyards, community gardens, and parks. I’ll keep researching those, getting involved and learning more where I can, but it will take patience and long-term effort.

So, in the meantime, I started thinking about other ways to help out. And my thoughts soon gravitated toward tools to help make the process easier. Already, there are some fantastic existing apps for finding plants endemic to your area — the Native Plants Database from the Lady Bird Johnson Wildflower Center, the Native Plant Finder from the National Wildlife Foundation, the Native Plants Database from the Audobon Society, and likely others that I missed. I would strongly encourage anyone interested in native planting to explore all 3 of those top-notch apps (and any apps I missed ) — each has its strengths and weaknesses. But something felt missing from all of them.

The missing ingredient was simplicity. These existing apps are phenomenal, but they produce long lists of relatively unsorted native plants —100s of results that would doubtless be extremely useful to an expert gardener/landscaper, but felt overwhelming for a novice like me just beginning to explore the possibilities. What about manageable suggestions for a newbie, starting small, with a few plants at a time? What plants could I fit in pots or a window box, within the limitations of my apartment? What if I sought specific characteristics in my plants? And when should I plant them?

What about a simple database, with an annual calendar and easy-to-use visualization tools for a newcomer to the field? It’s what I wanted more than anything. But after scouring the Internet for such a tool, I couldn’t find one. And as Toni Morrison once said on the craft of writing books: “If you find a book you really want to read but it hasn’t been written yet, then you must write it.”

The same goes for web apps.

Methodology

This app was an R project from start to finish: I gathered all the underlying data using a mixture of R packages for HTML scraping and FTP services, cleaned the data using mostly libraries within the Tidyverse ecosystem, and constructed the app using the Shiny web framework. Within the Shiny app itself, I made heavy use of the Plotly and Leaflet R libraries. I won’t go over the specifics — those are mostly covered in the app itself or in the Github documentation — but should note that the structure of the app was heavily influenced by the CRAN Explorer, a fantastic Shiny app that was one of the runners-up in the 2019 Shiny Contest. That app makes heavy use of custom HTML/CSS templates into which only selected R-to-HTML Shiny elements are fed, rather than the complete generation of all webpage elements from the underlying R that is commonly seen in beginner Shiny apps, allowing for the aesthetic flexibility that I required. The CRAN Explorer is also just a great app on the user-facing end, cool for not only looking at CRAN metadata but also discovering new R packages!

I learned a large amount from this process, but it would be tedious to go over all the little lessons I learned, so I’ll focus on a larger insight that dawned on me a few months into the project and which has stuck with me ever since. It’s a bit of a convenient narrative, a slightly-too-neat way to sum up the lessons of the app, but I can genuinely say that it’s a rule I’ll utilize on any subsequent web development projects.

In summary: just as back-end tasks are rendered easier by converting qualitative data (strings) to quantitative data (numbers, Boolean values), front-end communication is rendered easier by reducing user-facing info (text) to symbols (images, Sparklines, glyphicons). Alternately stated: wherever possible, reduce everything to symbols.

That principle is simple enough to understand on the back-end, and it’s basically tautological to remark that digital technologies function on, well, digits. Turning the impossibly complex analog world of reality into a massive amount of carefully calculated numbers, to form an imperfect but pretty good mathematical representation of the physical world, has worked nicely since humans started getting machines to do it algorithmically over the course of the past century (though not without a lot of extremely hard work from many folks much smarter than I am). It’s how computers function at the lowest level of bits and bytes, but it’s also a useful principle while writing a high-level, abstracted programming language like R; even a relative novice like me has learned pretty quick that data wrangling and subsequent calculation is better run on numbers or Boolean values than it is on a bunch of messy textual data. On the back-end, digitizing everything (where possible) is often best practice — and after a couple years of using R, I‘d internalized that lesson by the time I began work on my Shiny app. So I’d started with 1/2 of a guiding principle, sufficient to get started on a clunky but functioning back-end.

But on the front-end of the web app, I was completely lost. And that lack of experience shone forth not just on readability of my code (apologies to anyone who tries to scan my Javascript) or performance (I’m afraid to check any site speed metrics) but on the end product itself, the beauty and responsiveness of the user interface — the aesthetic of the site. I don’t have a visual eye — I am not a designer. The front-end was always going to be the biggest challenge for me, and I had no guiding principles beyond a general desire to make the site look nice.

But I knew I wanted to convey a lot of information in a neat, packaged fashion — a distinction from the existing native plant apps, which contain an enormous amount of powerful data but throw it at the user in long lists all at once. And a few months into the project, faced with a front-end that was improving but remaining stubbornly messy and text-filled, I had the idea that symbols on the front-end could help aid clarity and efficiency of communication in exactly the same way that numbers helped on the back-end.

Take a simple category (like Fire Resistance) with 2 possible values: “Yes” and “No”. Unlike other categories such as Fire Tolerance, which is measured in degrees of tolerance to fire (“Low”, “High”, and so on), the USDA has gathered only binary data on whether plants have Fire Resistance: they do or they don’t, “Yes” or “No”. Presenting those values to a user immediately creates a problem: the “s” jutting out of the “Yes”. It immediately increases the width of any column if that column’s value is positive. Thus, the challenge: how do you equalize the length of the Boolean values that you present to the end user, such that you can design the interface around columns that will be equally wide regardless of the positive/negative value? “TRUE” and “FALSE” present the same problem as “Yes” and “No”. You cannot present 1 and 0 to a user, even if you wish you could. Most folks would probably understand “Y” and “N”, but doubtless others would be confused. “Yea” and “Nay” are of equal length and so might serve, but your user is unlikely to be an aristocrat from the 17th century. And none of them are any good for a user who can’t read English.

But 2 symbols of equal size — in this case, the checkmark glyphicon and the x glyphicon — work perfectly for predictability of length and height in the design of the interface. It might seem obvious to those with front-end design experience, but it took a month of fruitlessly formatting a carousel in different ways before I arrived at that insight, the implementation of which took a couple hours and immeasurably improved both the readability and visual neatness of the carousel. Compact blocks of positive/negative categorical data for users, unmarred by an extra “s” protruding from the positive value: very tidy, with Lego-like ability to stack upon one another and lay neatly side-by-side without fear that the columns might be uneven.

The implementation of Sparkline to symbolize categorical degree data for a plant’s characteristics soon followed the Boolean data; “Low”, “Medium”, and “High” all have different lengths, but dynamically generated HTML inline charts don’t. And the charts are far more readable at a quick glance. The final effect of the word-to-symbol conversion is a cleaner, prettier interface that is also more efficient at quickly conveying data to a user.

The best part: these 2 approaches — a mostly numeric back-end and a mostly symbolic front-end— complement one another. For example, the R Sparkline package does not play well with non-numeric data. You can jimmy-rig it by messing with the underlying Javascript, but that can quickly become frustrating if you don’t know much Javascript. Better to just feed it the numbers that it wants, straight from the back-end.

Every symbol in this chart can be encapsulated in a uniformly-sized square, but you couldn’t do the same for their full word equivalents. Source: Wikimedia Commons

At the end of the day, anyone with a fraction more experience in either software or user design should feel welcome to roll their eyes at my prescription to reduce everything to the symbolic — yes, it is reductive. It’s not an inviolable rule, and obviously there are places where text is irreplaceable. But for this web app and any others I may build down the line, stressing the symbolic in both front-end and back-end — it’s a decent starting principle.

Momentum

There’s a ton of work yet to be done. The next step is to automate the inputs of the app — I’m currently setting up a Google Cloud VM to automatically run the R scripts that scrape/pull the various USDA/NOAA/NCSS plant, soil, and weather inputs and wrangle them into usable forms for the web app to use. I need to create improved weather models, to better anticipate the conditions in which each special endemic plant thrives — drier conditions for the tapertip onion, wetter conditions for the common persimmon. I need to find climate datasets for Hawaii, Puerto Rico, the US Virgin Islands, and the rest of the USA’s overseas territories. All these challenges and more remain.

On a technical level, I may need to begin looking beyond R as my basis for a user-facing web app, a task for which — despite the most valiant efforts of the brilliant folks behind Shiny —the language does not seem quite optimized in a truly scalable, durable way. Granted, part of that stems from my own weaknesses as an R user. They say that when all you have is a hammer, everything starts to look like a nail — and when all (or most) of what you have is Tidyverse R, everything starts to look like a dataframe. Everything is about applying vectorized functions to 2D matrices. That’s not necessarily a bad thing— as someone who entered R from a business analyst background, where spreadsheets rule supreme and Excel lives adjacent to 2D matrix management tools like Access and SQL, dataframes are about as logical a place to start as any. 2D grids are a powerful mental model that just about anyone can intuitively understand with little training, and their centrality to Tidyverse R is one of the reasons a novice like me was able to struggle my way into learning a little bit of the R language in the 1st place (along with — as any Tidyverse devotee will tell you — the magical syntactic sugar that is the %>% method and the linearity in coding that it fosters). Still, efficient programming relies on a bigger arsenal of data structures than the dataframe, and web apps may well require more flexible methods of coding. And more broadly — as great of a tool as Shiny is — it’s among the lightest web frameworks out there, lacking some of the powerful features common to Python or Ruby frameworks that might be useful for future improvements to the app, including email functionality.

Or maybe I will keep using R: optimize my existing code rather than re-write from scratch in a new language, take advantage of more Shiny features, and automate the back-end of my project. Whatever works best to keep the site up, running, and constantly improving. That’s the great thing about starting with a powerful motivation (like the desire to help folks plant endemic) and figuring out the methodology later on — you’ll keep on adjusting, tinkering, developing. In short, making it work.

So find a good motivation, a hobby, a skill — then find a way to scale it. If you happen to be a novice gardener, plant native! If you happen to be a novice developer, build a web app. And if you’re both? Do both.

--

--

--

Everything connected with Tech & Code. Follow to join our 900K+ monthly readers

Recommended from Medium

Engineers, Learn to Negotiate

Secure Software Supply Chain (S3C) in Serverless world

Developers, Developers, Developers: Microsoft/GitHub and The Ascendancy of Code

A simple, effective test automation strategy

Refactoring — The low hanging fruit

Making An Enemy AI That Can Detect When The Player Is Close

The 5 unit testing guidelines

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Liam A. Trotzuk

Liam A. Trotzuk

he/him. having fun with my friends in cyberspace

More from Medium

The best strawberries I’ve ever tried were right in this forest

Carla Boyd on Trailblazing Hemp & Technology

Can Hemp Help Us Save the Planet?

How to Find and Track Birds Worldwide with eBird

Indigo bunting.