Thar She Blows!

Be it out of steadfastness or out of stupidity, man has always strived for something that at the time probably could’ve been gotten a lot easier in another way. The thing that separates the latter from the former I think is the time spent on insignificant things. But what then constitutes significance? What makes one task more worthy than another on any one project? Should style be sacrificed for function? Well, surely if the damn thing has no funciton at all then style should be the last thing to be thinking of at that point. Just how far does it go though? If you’re trying to make an impression then style and how your web app looks plays just as big a role if not more so than what it can actually do. I have to admit that even if a site has dog shit for content but looks pretty and everything makes sense visually I’m going to think that site is a great website. At least they didn’t waste my time and hurt my life through my eyes.

But whether we’re thinking about app styles or chasing twisters in Nebraska the question remains. Just how far do you go? So I got it in my head after doing a couple of Python Flask projects that the next thing I did would be fast. My Flask project had used Beautiful Soup and though that library is great at what it does it isn’t fast. I knew that my app would be making a lot of calls to a database. The app was a project where I’d scrape the web using Scrapy for data on commercial and home standby power generators. Take that data, throw it in a MongoDB database and then interface with that data somehow for presentation to a user. That user could filter by company, model, special deals, etc. That was my idea and it’s pretty much done. It works. I’ve only got it setup for one company right now but more on that in a bit. So, fast it had to be fast. During my Google searching I came across the Tornado framework created by the people at FriendFeed. Also during my searching I kept coming across videos on how going asyncronous was the new “thing” basically. I figured sounds great!

Async IS great. But async IS difficult. Not neccessarily difficult to think about because from what I can tell it’s seems difficult for people who’ve already been doing things a certain way. Since I haven’t really been doing anything at all I got the concept pretty straight away. The hard part to me was sifting between examples and documentation that was Python 2 & 3. This is something I’ve run into before but with the Tornado framework it was at all time levels of occurances of things being shown in the Python 2 way of doing things. I cannot fully express how frustrating this is to someone who’s new to the language. This is pretty much the one and only knock I have on Python at all but it’s a big knock. The amount of time lost due to small syntactic differences between Python 2 & 3 is incalcuable. I suppose this is how the Python 2 people felt in general having 3 hoisted upon them but Python 3 happened, now let’s all use Python 3 and remember Python 2 fondly. Remember but never use, EVER again.

Let’s just say I spent a while switching from yield to await and back and back and forth. Why something would render a template with yield but then not await or vice versa. It got pretty confusing to say the least. The other gripe I have but it’s not really due to anything other than a lack of users, is the sparse tutorials or real solid good examples on how exactly to do things the “Tornado” way. I actually found much more stuff on AsyncIO than for Tornado and what’s weird is that almost all of it was done by the same people. It’s as if AsyncIO came along and development on or with Tornado was all but dropped. Since FriendFeed ended in 2015 that’s probably not wrong but that’s just how it seems from an outsider looking in and trying to learn the framework. The stuff I found for Python Flask by comparison was usually more hammered out or indepth and there was usually multiple different versions of the tutorial, guide or example. I’m afraid the same just isn’t true for Tornado. They do have documentation and some really great examples but then often you’re just left with a function definition and no example on how to use it or how it’s usually used.

After fiddling around a lot I felt I’d gotten to a point where I’d better start looking at my web scraping with Scrapy. This is where things got a bit easier and smooth sailing. A true Python experience really. I looked at the very nicely laid out website for Scrapy and poured over the docs and code snippets to get an idea of what I was looking at. It goes like this. Scrapy sends out your spyders, they crawl the specified url you’ve given them for the data you tell them to get. That data is funneled to an Item class where it get’s initialized. Then you send those Items with the data to a pipeline where you do anything to it like say string manipulations and the like. Then on into your database. That’s it. There’s a little more to it but really in all honesty not much. That’s what’s so great about Scrapy and Python in general. It just works, usually.

When it doesn’t it’s often because of something you or more to the point, I, am doing wrong. This is definitely the case with Tornado, despite what I’ve said on the shortcomings of the documentation & examples most of my issues were in fact due to my plain old inexperience with creating and then using restful API’s. Much of what web development I’ve done throughout the years were small business sites, artists, musicians, that kind of thing. Not a lot of data being thrown around, mostly HTML & CSS really with some Javascript and jQuery here and there but not much outside of my short stint working for an online streaming media company. Enabling CORS and then going through the whole routine of setting headers and making sure things were just so took some time. I’d messed with CORS before but only little scripts for Wordpress sites, just setting some config file basically. Nothing like actually creating the settings on a Python back end. Pretty cool.

Then though there’s ReactJS. Out of all the different ways of doing things on the front, React has gotten me excited the most. I’ve already talked a bit about why that is in my first blog post to the site so I won’t repeat myself here. But updating this site today with Gatsby was just really really fun. The only knock I have on React so far is that since it’s so new still things in the React world are still being figured out. One thing React does suffer from, just like Tornado, is the amount of bad examples floating around for it. It’s only been around for a couple of years and still there are big differences in how to do things from when it began to now. They aren’t HUGE changes but ones that if you aren’t aware of will drain a good amount of time from you. When you do discover the differences you’ll ask just why in the hell the old references are still kept around? One key example is just why in the hell do they keep the deprecated PyReact library ON the React website? I understand the Python library will be floating around forever but do you have to list it on the site as if it’s still relevant? If you look at the page there’s nothing telling you that it’s no longer valid or that it shouldn’t be used. You’re just kind of left to figure that part out for yourself.

I think I’ll actually start wrapping this up and will split this into two or perhaps even more parts as this is already getting pretty massive in length and I’ve only lightly skimmed my trials and tribulations with my little power generator info tool. So check back for part two!