Anyway, as part of a new feature I’ve been building, there’s a method called aggregate_trip_view. What does it do? It doesn’t really matter. The point is, there’s a loop in there where one of the steps is

bus_line = BusLine.find_by(line_ref: line_ref)

I had noticed this particular method was pretty slow. I had put some timers in the logs, and was getting results like

7.63 seconds… Why was looking up BusLines so slow? I had a theory, since I’ve come across something similar on this project before.

To test it, I ran

Rails.logger.level = 0

This set the logger to `DEBUG` level so that I could see every database query.

And sure enough,

Rails was running a different SQL query every time I looked up a BusLine. I only have 334 BusLines in the database, but since it’s a separate query for each one, it adds up fast!

So, how to solve it? Of the very few things I know about computer science, one of them is: Hash maps are fast. So I decided to try using a Ruby hash to help me out.

Instead of using Rails Active Record to look up 334 individual BusLines, I refactored the method like so:

# Outside the loop
bus_lines = BusLine.all.each_with_object({}) do |bus_line, bus_lines_obj|
  bus_lines_obj[bus_line.line_ref] = bus_line
end

This uses each_with_object and makes a hash (Ruby object) where the key is the line_ref (bus line ID), and the value is the individual BusLine instance. The upside of this is that it’s only one single SQL query (SELECT * FROM bus_lines) instead of 334!

Now, inside the loop, I can look up the BusLines much more quickly, like so:

# inside the loop
bus_line = bus_lines[line_ref]

And sure enough, the results were much better!

Over 7 seconds down to 0.05. Not bad!

ui: {
 loading: true,
 feature: ‘BusLines’
}

When loading is true, my React component would display the spinner. When loading changed to false, the component would see that and remove the spinner. The feature key is just so I would know which type of data I was waiting on (I have a few, including BusLines, RealTimeData, and HistoricalDepartures.) And of course, this worked great! Until it didn’t.

My spinners worked, but they always seemed to be disappearing too soon. I would see the component appear with blank data, and then the data would populate a second or two later. Why was this happening?

You may already see the problem: What if we’re loading more than one type of data at once?

The request would start for RealTimeData but I would also need some HistoricalDepartures. Now both requests are pending at once. When the first one comes back, it was setting the loading flag to false, causing my spinner to disappear prematurely.

At this point it was getting more and more apparent that I needed a better solution. Once you have more than one type of data, it’s not enough to keep track of whether I’m loading something. I also need to know what is loading, and be able to handle multiple things loading at the same time. So what should our little corner of the Redux store look like?

The first solution I was thinking about looked something like this:

ui: {
 loading: true,
 features: [‘BusLines’, ‘HistoricalDepartures’]
}

Here we use an array to store which features are loading. That way, I’d be able to keep track of the loading state of multiple features and not just one. But what do we do when, say, the BusLines request comes back and is no longer loading?

# ui.reducer.js

let newFeatures = ui.features.filter((feature) => feature !== ‘BusLines’)
let newLoading = (ui.features.length === 0)

This works just fine. Now what if BusLines is still loading, but we also just started loading RealTimeData?

if (!ui.features.includes(‘RealTimeData’)) {
 newFeatures.push(‘RealTimeData’)
}

Also works okay, but it’s just a bit inelegant. After all, we have to iterate through the whole array just to make sure we’re not duplicating elements.

In the end, instead of an array I decided to use Sets to solve the problem. I think Sets are a perfect solution for this situation.

So now our corner of the Redux store looks like this:

# default state
ui: {
 loading: true,
 features: new Set()
}

Now, I can do this

features.add(‘RealTimeData’)

as many times as I want, and the Set won’t have any duplicates.

I can check for elements like this:

if (features.has(‘RealTimeData’)) {…}

I can remove elements like this:

features.delete(‘RealTimeData’)

(Note: This will work even if the set doesn’t include that element. It won’t throw an error; it will just return false instead of true.)

A couple caveats (don’t worry, they’re not too bad):

• Sets are part of JavaScript ES6, so they can’t be used in older JavaScript versions.
• If you use Redux Devtools extension, the value of the set doesn’t display properly in the extension. But rest assured it’s storing the data just fine. You can confirm this by doing some console.logs, which are actually kind of fun to watch:

After implementing my loader state as a Set instead of an Array, it worked perfectly. Try it out next time you need to store some unique values and you don’t care about the order!

More info: https://alligator.io/js/sets-introduction/

JavaScript:

function birthdayCakeCandles(ar) {

  var max = ar.reduce(function(a, b) {

  return Math.max(a, b);

  });

  var count = 0;

  for (var i = 0; i < ar.length; i++) {

    if(ar[i] === max) {

    count++;

    }

  };

  return count;

}

Ruby:

def birthday_cake_candles(ar)

  ar.count(ar.max)

end

Both of these solutions do exactly the same thing. It’s striking and just a little bit hilarious how much more elegant and simple Ruby is.

(If you have a one-line JavaScript solution that I’m missing, I’d love to see it!)

… Install this Atom package here. It fixed it for me!

000-project-shell-env

It’s something to do with environment variables. The related Github issue is here: https://github.com/atom/github/issues/764

Basically we have to do three things:

1. Gather our data
2. Prepare and assemble the data
3. Encrypt the data

Sounds simple, right? As usual, no. Welcome to OAuth 1.0.

Gather data

This is the simplest part. We need the following stuff to be part of the signature:

• The HTTP method (usually ‘POST’)
• The URL we’re sending the request to
• oauth_consumer_key: Our Twitter API key
• oauth_nonce: A string value which uniquely identifies the request
• oauth_signature_method: ‘HMAC-SHA1’ is the hashing algorithm which is used to encrypt the signature. Ironically, we have to tell Twitter that we’re using HMAC-SHA1 to encrypt the signature, but this information can only be viewed after decrypting. Okay.
• oauth_timestamp: The number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, January 1, 1970, minus the number of leap seconds that have taken place since then. Don’t ask.
• oauth_token: The access token we got from Twitter (see the previous post)
• oauth_verson: ‘1.0’
• In addition to these key-value pairs, we also must include any other key-value pairs we’re sending as part of the request.

Prepare and assemble the data

For each of the key-value pairs above, we have a list of stuff we have to do:

1. Percent-encode it. This is Twitter’s doc explaining percent-encoding in detail. Long story short, we turn a string like

Dogs, Cats & Mice

into something like this:

Dogs%2C%20Cats%20%26%20Mice

2. Sort the list alphabetically by key, except that the HTTP Method and URL go first.

3. Assemble the string according to the format “key=value&key=value” (except that HTTP Method and URL don’t include key=)

Once that’s done, we have a Signature Base String.

Encrypt the data

In order to take the parameter string and turn it into a signature, we need to get a Signing Key. This is done by assembling two values, the Consumer Secret (Twitter API key) and OAuth token secret, if we have one (obtained from Twitter by doing the sign-in dance explained in the last post). The Signing Key is assembled using the format “value&value” (or, if we don’t have an OAuth Token Secret yet, just “value&”.) (By the way, we also have to percent-encode these before assembling them.)

Now that we have our Signature Base String and Signing Key, there are only a couple steps left!

1. Encrypt the Signature Base String with HMAC-SHA1, using the Signing Key as our encryption key. This will give us a binary string, something like
   84 2B 52 99 88 7E 88 7602 12 A0 56 AC 4E C2 EE 16 26 B5 49
2. Encode the encrypted string using Base64. This will turn it into something like
   hCtSmYh+iHYCEqBWrE7C7hYmtUk=

And we’re done! That’s our Signature.

By the way, the values used above are example values from Twitter’s documentation. This is the only way Twitter gave me to test if my code is working correctly. Using the example values from Twitter’s document, if you do all the steps correctly, you should end up with the signature above. If you get even one parameter incorrect, or leave out one key-value pair, you’ll end up with a completely different signature and Twitter will reject your request. In that case, there are no error messages to help you find where you’re going wrong. Twitter will just send back a cryptic error. You just have to think a lot and keep trying. (Note: Make sure your signature base string doesn’t have any leading or trailing spaces. That one took a while for me to catch.)

Thanks to Wikipedia for the definition of Unix epoch.

To implement Sign in with Twitter, Twitter requires that I use OAuth. The benefit of using OAuth is that my app doesn’t need to know your Twitter password, but you can still use your Twitter credentials to log in. However, for whatever reason, Twitter is still using OAuth 1.0a for this process, not the newer OAuth 2.0. This means we have to do a very intricate dance of AJAX requests between my app and Twitter’s API:

1. Send a signed POST request to api.twitter.com/oauth/request_token. This request contains a callback_url, which Twitter will use later in Step 4. If everything goes OK, Twitter will respond with a token and token secret.
2. At this point, my ‘Login with Twitter’ button becomes available. When it’s clicked, the user is directed to api.twitter.com/oauth/authenticate, and the token from Step 1 is included in the URL parameters.
3. Now, Twitter will handle the login. If the user hasn’t already authorized my app, Twitter will display its login screen and the user can enter credentials. If that’s done already, Twitter will just skip to the next step.
4. Once login is complete, Twitter will actually redirect the user back to my app’s callback_url, which I specified in Step 1. The URL parameters Twitter sends me will contain the same oauth_token I got back in Step 2, and an oauth_verifier which I can use in the next step.
5. When I get this response back, I must send another signed POST request, this time to api.twitter.com/oauth/access_token. The request will contain the oauth_verifier I obtained in the previous step.
6. If the planets are aligned properly, I’ll finally get an oauth_token_secret in the response. This is my digital wristband that proves that I’ve been admitted to the party. I should be able to use the oauth_token_secret for future Twitter API requests without having to do this dance again.

Think that’s enough complication? Not even close. There are at least 3 things that add complication to this process:

1. All the parameters must be percent encoded, properly, or it won’t work.
2. All the requests I send to Twitter must be signed. A signature is a long string containing all the other parameters of the request, percent encoded, sorted in a specific way, and then encrypted using HMAC-SHA1. Maybe I’ll go into more detail about that in a future blog post.
3. If I get something wrong, Twitter’s API responses are quite opaque. They definitely don’t help much.

I handled most of this using my Ruby on Rails backend, though of course my React front-end had to handle the part where Twitter redirects the user back to my app. I was able to pull out the URL params using the React Router location props, which I mentioned in my previous post.

Here are the three main Twitter documents I used, that explain all of this in excruciating detail:

Implementing Sign in with Twitter

Creating a signature

Authorizing a request

I did use the Ruby OAuth gem, but only for the OAuth::Helper methods to percent encode my strings properly. (I also used its parse_header method to turn the incoming params into a hash.) But that’s it. I didn’t use any of its built-in methods for creating signatures or obtaining tokens. This is partly because I found the documentation difficult to understand, and partly because I knew it would be a great learning exercise to do it all manually.

Like, don’t get me wrong — it’s great. You can put your routes in just like any other React components, you can nest them, the design is elegant, and in an ideal world, it “just works.’ But… The documentation leaves a lot to be desired. It seems like they make simple tasks feel complex. I’m working on a project using React Router this week. The last time I used it was several weeks ago, so naturally I needed a little refresher. I just had some basic questions, like:

How do I prevent my entire app from remounting?
I don’t want anything fancy. I just want to say, ‘If the user goes here, render this.’ How do I make simple routes that make sense?
How do I prevent React Router from matching multiple routes and rendering too many components?
How do I make my NavBar work right?
How do I make buttons work well for navigation?

I had a lot more trouble answering these than I feel like I should. So I thought I’d write this post in the hopes it helps some React Router beginners like me. Here’s a few things I’ve learned:

If you use the URL bar in the browser, your entire app will re-render. The consequence of this is that your app state will be obliterated, including any data you had loaded. At first this threw me off a little, but it makes sense. If you type an address and press enter, your browser does a new GET request, and there’s probably no getting around that. (If there’s something I’m missing there, I’d love to be corrected!) So how to get around this? So far I’ve found two ways: First, use the <Link> component provided by React Router, instead of regular <a> tags. I’m not really sure how they differ, but it seems to prevent the app from reloading. But what if you want to have a <button> such as ‘Create User’ that takes you to a different page? I don’t know how ‘correct’ this is, but what I’ve been doing is

1. Have an onClick event handler for the button which sets a redirect flag to true.
2. In the component’s render method, check if redirect is true. If so, render a <Redirect /> which is a component provided by React Router that will send you to whatever route you specify.
3. After this is complete, you have to remember to set the redirect flag back to false, or else you’ll be caught in an infinite loop. I think React’s componentDidUpdate lifecycle method seems like a good place to do this.

Update: None of the above is necessary. I found out that this.props.history.push is a great way to get your buttons to work right! You just have to use it in conjunction with withRouter, explained below.

Another thing beginners like me have trouble with is the React Router v4 philosophy. Instead of thinking in terms of simple routes, you’re supposed to think in a more complex way about how you want your app to display. The basic idea is conditional rendering: Given any Route, if the path matches, render the component. If the path doesn’t match, don’t render it. That means that React Router does some things you may not expect.

For example, let’s say you have a group of routes like this:

<Route path=‘/users’ component={UserList} />
<Route path=‘/accounts’ component={accounts} />
<Route exact path=‘/users/delete’ component={DeleteUserForm} />

If the user navigates to /users/delete, which components get rendered? You may think it’s just DeleteUserForm, but you’d be surprised. Both /users and /users/delete contain the string ‘/ users’, so this would match both Routes and both components would be displayed. This is true even though the DeleteUserForm specifies the ‘exact’ prop. Exact doesn’t prevent other Routes from being rendered — it just prevents that specific Route from rendering if it’s not an exact match. Thus, if the user goes to /users, only UserList would be rendered because ‘/users’ is not an exact match for /users/delete. But if you navigate to /users/delete, both routes match.

(By the way, to make React Router behave a little closer to standard expectations, you can enclose a group of Routes inside a <Switch>. React Router will render only the first Route that matches, and will ignore any subsequent routes in the Switch.)

To add even more complexity, here’s something else to think about: I’ve been saying certain things ‘don’t render,’ but that’s actually not the case. React Router actually renders every Route, every time. It just renders null if it’s not a match.

Ok, one more thing: React Router gives its Routes some handy props that give you some useful information:

match: How we got here. this.props.match.path contains the url that was actually navigated to. match.params contains any URL parameters (the stuff in the URL after the ?.)

location: Where we are now. this.props.location.pathname contains your current location.

history: The complete history object.

The thing to remember about this is, only the Routes get these props. Your other components won’t have them. But what if we want that info to be available to, say, a NavBar so we can keep track of where we are? To do this, React Router includes the withRouter method. It’s easy to use. Say you have a NavBar React component, declared as an ES6 class:

class NavBar extends React.Component { … }
export default NavBar

All you have to do is wrap it in WithRouter, like so:

export default withRouter(NavBar);

Oh, and don’t forget to import { withRouter } from ‘react-router-dom’

Now, your NavBar component will have those props.

Hopefully this post will be helpful to some React Router beginners like me. If you have any tips to share I’d love to hear them!

If those two paragraphs weren’t there, I guess the article or blog post would feel a bit weird, but 99% of the time I find myself skimming right over that part of the page anyway. Especially with clickbait headlines, the “answer” to the question doesn’t present itself until a little farther down, and with experience it’s easy to make educated guesses as to where that answer is and just jump right there.

Scoping is hard

Before you start your project, you think things like “I can easily finish authentication, 16 models, 9 pages, and 3 API integrations in 3 days!” Then you learn that this is not the case. I feel like it’s going to take me several more projects before I’m good at accurately estimating how long things are going to take. I’m happy that instructors are going to help us with scoping our final projects, which are longer than anything we’ve done up to this point.

It’s a project, not a product

It’s easy to think that your project has to be an amazing business idea. It has to have a great domain name, you should think about how to market it, you have to consider how well it will scale, if it’s been done before, network effects, customer acquisition, etc. NO! Your project is a project, not a product. Unless you’re here in between semesters at business school (sorry, Paul) the point of doing a project is to show your proficiency in Rails, or React, or JavaScript, or Ruby — in other words, to showcase your skills as a developer. During our limited time here, it’s best to keep that technical focus. There’s nothing wrong with great business ideas, but I feel like they can wait until you have more time.

It does matter how it looks

With that said, the other side of the coin is that appearances ARE important. You can have an amazing back-end with some Einstein-level logic happening, but if your site looks like it’s from 1995, it’s just not going to be as impressive. There are a ton of CSS and other frameworks out there (Bootstrap, Materialize, Semantic) which make it REALLY easy to make your project look decent, fast. And it’s totally worth it to put in the time to implement one of them. In my experience, it actually makes a HUGE difference in how impressed people are with your project. If you take the time to make your project look great, people will appreciate all your hard work on the back-end logic. If you don’t, they won’t. It may sound harsh, but it’s the truth.

It makes it click

Over the past twelve weeks, we’ve become all too familiar with the Flatiron School process of Learning a New Thing:
1. Do a bunch of labs on Learn.co. Barely grasp the concepts. (#$%^& tests!!)
2. Listen to lectures. Seems easy.
3. Try to do it. Nope.. it’s hard.
4. Repeat steps 1–3 for two weeks.

But then when you get into Project mode, it all changes. Project mode is when you actually get a lot of practice doing the new thing. A lot of the time, it’s also when the new concept finally ‘clicks’ for me. By the end of my project, I actually feel confident using my new knowledge. I feel ready to take the code challenges. And I feel like I could do it all again and not make all the rookie mistakes.

You see your dark side

Project mode is when you can have some of your most challenging moments. I spent a LOT of time frustrated and stuck. I found out the hard way which parts of the New Thing I really understood, and which parts I didn’t. I found out the hard way how I can have little moments of tension with others in my group, and I always felt like I could have dealt with it a bit better. I found out that doing projects is a great way to get a sense of what your weaknesses are, as a programmer and as a person. This is stuff that labs and lectures just can’t give you. And though it’s difficult to deal with, the personal growth potential is one of the things that makes it worth it.

But in Chrome, it wasn’t working at all. I would get errors like this:

What the heck is an ‘Access-Control-Allow-Origin’ header? And why do I need one? And why isn’t it there? And if I really need one, why does it work on Safari!?

(That last one I still haven’t answered. But anyway.)

I wanted to find out how to resolve this. After consulting with my instructors at Flatiron School and plenty of googling, I discovered lots of different resources online trying to help people resolve this problem. But I noticed something strange. Pretty much all of the posted solutions assumed that you were the owner of the API; that you controlled the server and could make changes to it. See, all you have to do is send back an ‘Access-Control-Allow-Origin’ header of ‘*’ and it will work great!

But alas, I am not the New York Metropolitan Transportation Authority, nor do I control their servers. So what’s going on?

Turns out that CORS is a standard enforced by the browser, and it doesn’t allow cross-origin requests unless the ‘Access-Control-Allow-Origin’ header says so. So what’s the difference between a cross-origin request and a same-origin request?

Let’s look at this example:

Server sending the request: bustime.mta.info
Server receiving the request: bustime.mta.info

The above would be considered a same-origin request.

Server sending the request: www.mywebsite.com
Server receiving the request: bustime.mta.info

This would be considered a cross-origin request.

So, let’s say the MTA has its own internal web app at bustime.mta.info. That web app can send same-origin requests to its backend API all day, and CORS won’t even be involved. This setup is normally used for API’s that are for internal use only — in other words, not public.

If the MTA has another web app that’s at a different URL (not bustime.mta.info), they can set their ‘Access-Control-Allow-Origin’ header to allow requests from that specific URL. They know where the request will be coming from, since it’s their own app, so it’s just a matter of configuring their API to allow that request.

But, you might say, that’s fine for an API that doesn’t send out data to outside developers. But this is supposed to be a public API. Why block cross-origin requests on a public API!?

I was wondering that too, for the longest time. And then I found this post in the MTA Developer BusTime Google Group. It says

“The MTA’s data feeds are provided without charge to developers who agree to the following:

1. In developing your app, you will provide that the MTA data feed is available to others only from a non-MTA server. Accordingly, you will download and store the MTA data feed on a non-MTA server which users of your App will access in order to obtain data. MTA prohibits the development of an app that would make the data available to others directly from MTA’s server(s).”

It goes on to suggest that instead, you must provide your own back-end server to make the requests and download the MTA feed, and then pass that data on via your own API feed. Since it’s not a browser making the request, CORS will not be an issue.

So alas, it appears I won’t be getting around this issue any time soon. I guess it’s time for rails new — api.

Over the course of my several weeks here, I’ve had the opportunity to do pair programming (also programming in groups of three) quite a bit. People seem to have plenty of opinions about it. I didn’t think I was going to like it at first, but after trying it out I’m pretty much sold.

In case you’re unfamiliar, the basic idea of pair programming is this: One laptop, two people. The person doing the typing is called the ‘driver.’ The person watching is a kind of supervisor or ‘director.’ The ‘director’ thinks mostly about the ‘big picture,’ telling the driver what to do. The driver actually writes the code and thinks more about the small details such as syntax, etc. Ideally, you switch roles after a while so that everyone spends equal time in the driver’s seat.

So, after several weeks of this, I thought I would offer some observations, based only on my experience so far at Flatiron. What does pair programming do?

1. It gives you super powers.

In my experience, pair programming is GREAT for catching the little things. When there are two pairs of eyes on the code instead of one, stuff like typos and syntax errors are caught almost instantly. Even more often, you prevent the error from being made in the first place. Every time this happens, you may be saving yourself several minutes to an hour of debugging later. If you think of it that way, pair programming saves a ton of time!

Super power #2: You think of solutions you never would have thought of otherwise. You may have a habit of doing something one particular way, while your partner may have a completely different habit. You may realize that your partner’s habit is way more elegant and appropriate than yours, and end up with something new to add to your own bag of tricks. (Als0, you may learn a new keyboard shortcut!! Keyboard shortcuts are the best.)

Super power #3: When it’s 4:32 pm and you’re mentally exhausted and distracted and you can’t think, pair programming can be a lifesaver. Your insurmountable problem may be a simple solution for your partner, who just solved a similar problem the other day. Also, mental tiredness can slow you down quite a bit, and having a partner will make things that much easier.

2. To learn the most, work with people just a little better than you.

From my previous years of experience playing jazz piano, I had the opportunity to play in lots of different music groups with different people, and this is something I noticed. If you perceive that your group-mates are WAY better than you, you may feel like a burden, or feel overwhelmed. (Not that you should.) Conversely, if you perceive that your partners are much LESS experienced than you, you may feel a bit bored, or even annoyed. And then there’s the ‘Goldilocks’ of ability levels, when you are all very close in ability but you feel like your partners are just a little better than you. If you’re lucky enough to have that experience, it feels like you’re just barely hanging on, but you ARE hanging on. And learning a ton.
Even better, if your ability levels are ‘Goldilocks-matched’ and you have a great, respectful relationship with your partners, you may even come across a sort of M.C. Escher-style “cycle of respect” where EACH member of the group feels the same about the others!

This is barely even an analogy. I think it’s safe to say that all of the above can apply directly to pair programming in addition to music.

3. It gives you an opportunity to get better at SOMETHING, no matter where you are on the ability spectrum.

If you’re the less experienced partner, you will get better by driving. That goes without saying. But what if you’re the more experienced person? What if you know all the required concepts and could easily finish coding this by yourself? You STILL have plenty you can learn by teaching your partner, and making sure they understand. Even if you don’t think you have any gaps in understanding, teaching someone is a great way to find out for sure! It can also be an exercise in patience, speaking clearly, and grokking where the other person’s understanding gaps are. All of these are great skills to learn, even if you already know all the code!

So either way, you’re going to get better — at doing, teaching, or both.

Oh, and be nice! Everything works better when people are nice.