My Code Needed a Comment to Compile

 When I was learning python, I added a utf-8 string literal to the source code as follows:

# replace invalid ’ character with '
inputFileContents = inputFileContents.replace('’', "'")

Which caused this error:

SyntaxError: Non-ASCII character '\xe2' in file gcnlsync.py on line 85, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

My python code would not run until I added a comment to the file that declared utf-8 encoding. This is literally the top 2 lines of my file:

#!/usr/bin/python
# This Python file uses the following encoding: utf-8

After adding the comment, it ran perfectly! 😳

Raspberry Pi Apartment Hunter

Many apartment complexes use sophisticated algorithms to change lease prices on a daily basis. If they can use an algorithm to maximize value, I can too!

In this post, I will walk through the steps I took to grab data from an apartment leasing website, monitor changes, and send me email alerts.

Grabbing the Data

In this example we will take a dive into a Leasing Website in Glendale

A click on "Apartments + Pricing" will reveal a page that looks like this:

The web page is retrieving the apartment data from a remote server. Let's see if we can find it.

Using the Google Chrome web browser, select "View -> Developer -> Developer Tools", and refresh the web page.

In our Developer Console, we see dozens of javascript executions and connections to remote urls. One of the urls is called "ApartmentSearch". It contains all the beautiful apartment data we want. 

https://api.avalonbay.com/json/reply/ApartmentSearch?communityCode=CA068&min=2200&max=3400&_=1480230523132

Copy/paste the url into a new browser tab, and you'll find a handy, if messy, wall of JSON text.

Copy all the text. Let's visit jsonlint.com and pretty it up:

Fantastic! Scrolling through this data, we can find every apartment and its current lease price.

Monitoring Data with a Raspberry Pi

The website gives us a JSON object, which is very easy to read and manipulate in JavaScript. I wrote a server-side JavaScript script, "apartment.js", that parses the JSON data, and writes to a CSV file. Later, we use the CSV file to make pretty graphs in Microsoft Excel.

Here is a screenshot of the code parsing the JSON object:

When the data is saved to a CSV file, it looks like this:

I used a Raspberry Pi computer to run my "apartment.js" script every 6 hours. Each time the script ran, it would parse the JSON object, and add the current apartment prices to the CSV file. Server-side Javascript is possible thanks to the NodeJS project.  Raspberry Pi is an inexpensive computer that is popular for open-source projects. What I like most about a Raspberry Pi is that I can leave it on all the time without worrying about an expensive energy bill.

Running a script every 6 hours with crontab

Here I log into my Raspberry Pi and execute "crontab -e". This can be done on any Unix-like machine such as Mac OS or Linux

Inside cron, the bottom line runs the "apartment.js" script every 6 hours.

Pretty Graphs

What did we learn?



Between September 13th and October 7th, apartment 106 bounced around in price. Eventually, it slid to a sweet $2150 before being sold. Whoever claimed it got a great deal.



Between September 13th and October 16th, apartment 109 slid to an even cheaper price at $2070. However, after 106 sold, it quickly shot up in price and stubbornly stayed there. 109 was sold on October 16th for $2200. Whoever bought it missed the optimal deal.



Apartment 117 appeared on November 4th, and quickly shot up in price. It's been more than 20 days and no one has bought it...

Later, I will share more graphs if I discover any interesting trends.

Email Alerts

As the Raspberry Pi collected data, I had it send me emails when something interesting happened:

1. When any apartment reached its cheapest price ever
2. When any apartment became vacant or sold

This way, I didn't need to refresh the leasing website like a crazy person. 

Side note:

I implemented this algorithm for myself, so I took the lazy route and stored data in a text file for a single user. For a more scalable solution, I would recommend a MongoDB database. I chose to skip installing MongoDB at the time because the official MongoDB binaries did not support my Raspberry Pi.

Cheapest Price Ever

Prices dropped in September and October as vacancies opened up:

Eventually, a one-bedroom apartment bottomed out at $2070

Vacancies were sold and prices have climbed ever since.

Apartment Vacancy Changes

Starting on October 16th, I programmed the Pi to email me about vacancy changes

My email archives tell me a lot about how often vacancies were added or sold:

Notice that vacancies changed every 5-10 days until the middle of November. Since then, vacancies haven't changed for more than 20 days.

New apartments usually would popup with with much-higher-than-average prices. 😅

Source Code

I uploaded the source code onto GitHub!

https://github.com/KWarp/Apartment-Hunter

Conclusion

My Apartment Hunting algorithm confirmed the obvious. When lots of apartments are vacant for a long time, their prices slowly drop until they are sold. After one sells, the remaining apartments spike in price for a while.

The email alerts were certainly handy. I didn't need to worry about refreshing the web page every day for the best deal. With my Raspberry Pi collecting data directly, the apartment company could not track any web browsing data about me. That in itself was its own reward.

Video Game Prototyping

Video Game Prototyping is an emotional roller coaster. Two months ago, I started hammering at a totally amazing game idea. Every day was a coding frenzy. I couldn’t wait to get the game in people’s hands.

After a week, the first playable was ready. Players were excited by the gameplay, but wildly confused by the touch screen controls. I took a step back, and simplified the game controls from 8 buttons to 3 buttons. As a result, the game lost some of its strategic depth.

Another playtest took place. Players found the game easier to play, but also less interesting. Feeling humbled, but not discouraged, I set out to find the new fun factor the game needed. I spent days churning out one game mechanic after another. I felt completely in my element, putting all of my skills to the test.

As the days went by, none of the mechanics stuck. I felt this sinking feeling that maybe I can’t figure this one out. Each day became more difficult, each mechanic less fun, and after 8 days I ran out of ideas. My confidence hit an all-time low. Why was I even wasting my time on this?

The next day, an innocent thought popped into my head. The Bullet mechanic was fun, but lacked strategy. The Bomb mechanic was strategic, but too slow. What if… the bullets were also bombs? And a skillful player can strategically detonate them?

My confidence started climbing. This tiny layer of strategy enhanced every aspect of the game. Play time jumped from 2 minutes to 10 minutes. Laughing and trash talk ensued. The game was back on track, and even better than before.

This is an ongoing project. My mind is buzzing with ideas. Hopefully I can share it with you all soon!

Unity: Rotate a 3D ball using 2D Physics

Unity is an awesome game engine with great 2D and 3D features. While working on a 2D game world with 3D game art, I ran into an unexpected challenge with rolling 3D spheres. Let's frame the problem in the context of a 2D soccer game with a top-down view.

TL;DR
Before (animated gif)
After (animated gif)
GitHub Project

Crypt of the Necrodancer: Dance Pad Setup Tutorial

Crypt of the Necrodancer is an amazing game. I love the rythm-driven mechanics, and how it lends itself to play on a Dance Pad. While my reflexes are best with the keyboard Arrow Keys, I bought the Deluxe D-Force Dance Pad for a bit of exercise, and novelty at parties.

Rapid Physics Beta Release

Rapid Physics is a custom Physics Engine that I decided to build in Flash.

Why would I torture myself like that?

There are plenty of industrial-strength physics engines out there, such as Havok and Box2D. They offer complete and robust physics simulations, but they also hide a lot of information from the programmers that use them. This leads to two problems:

1. Hard limits on authorship control over the physics simulation (lots of code you didn't write)
2. A sense of helplessness when debugging engine-level problems (lots of code you didn't test)

Effectively, game programmers lose the granularity to control subtle features of the physics simulation, and the ability to do custom collision resolution for gameplay purposes. For games whose physics are integral to gameplay, there is still a place for custom physics implementations.

I designed Rapid Physics to tackle one specific problem that most general-purpose physics engines never attempt to address: Tunneling

Rapid Physics

I am building a physics engine called Rapid Physics. It is designed to correctly simulate objects that move really fast. This is difficult problem that is commonly known as the tunneling problem.

Check out a quick demo here:
Rapid Physics 0.25 Alpha