So this weekend, I wanted to spend some time on gathering some
statistical data from the Iracing interface via automated scrapping.
Here is the initial report, lets cut the suspense for the ones
who are reading :)
So the ones that are not familiar with Iracing license model it can
be explained as simple as:
When you join Iracing, you start with a rookie license, you can
get promoted to the next level license only if you complete
certain # of races, and meet a safety rating 3/4 or more.
As of right now, nobody had gathered and shared this statistic
of what % of users are in which license level as far as I know
and thats why i wanted to take on the challenge just to keep my coding/scrapping skills fresh as a small weekend hackathon :)
So How did I do it?
Well, when I started on the idea Friday night, i was quick enough
to find out, that you cant scrape using conventional scripting methods with python (mechanize and beautifullSoup). And yes I do enjoy scripting these types of things in Python :)
I used to use a similar tool back in the day called OpenKapow. But it had a
very extensive ETL like browser IDE.I personally feel I have more control over with Python compared to using a bulky IDE...exception handling is also easier this way.
+1 for Selenium folks! :)