Posts

Showing posts from July, 2014

Scrape Hidden Public APIs using C#

Image
So you found a hidden unprotected API of a website you like.
Lets use dynamic objects to represent JSON objects and save them in our database. For later processing. There are legitimate reasons to do this. Say a crappy built in search system of the website.



Please note that following code totally depends on the website backend implementation. Do your research and change accordingly.

Post might feel like out of sequence. It was intentional.
lets begin:

const string ajaxUrl = "http://coolsite.com/some-stuff?page=";
Lets create a separate method to issue the http request.

private async static Task<int> GetResponse(HttpClient client, string url, int page) { int pageCount = 0; var response = await client.GetAsync(url + page); response.EnsureSuccessStatusCode(); string responseBody = await response.Content.ReadAsStringAsync(); var serializer = new JavaScriptSerializer(); // here we need to register our own Dymaic JSON converter. //…