Apr 29: Using PHP and cURL to scrape ID numbers from search results
Very leading title, that one, I know. Basically, I' ve done this recently and I want to let people know how.
Sometimes you find that there's a repetitive data collection task that needs to be completed on the web, and you establish a pattern to doing it:
Fine if you're only doing it say, 10 times. Or even 20. I needed to do nearly 3000 searches like this on Amazon, when I was looking for ASIN codes for a stockroom's-worth of CDs. There was no way I was going to do it manually...
Read on for the code and an explanation. Read More
Sometimes you find that there's a repetitive data collection task that needs to be completed on the web, and you establish a pattern to doing it:
- Surf to the website's search page
- Type in the term, and select other form options to limit the search (a category in the case of a directory search, maybe?)
- look through the returned page for the corrrect result, and copy what you want
- paste that into a row on your spreadsheet (or whatever)
Fine if you're only doing it say, 10 times. Or even 20. I needed to do nearly 3000 searches like this on Amazon, when I was looking for ASIN codes for a stockroom's-worth of CDs. There was no way I was going to do it manually...
Read on for the code and an explanation. Read More
Apr 19: Misterhouse and Asterisk PBX
In case it is of interest to anyone, I've hooked up the MisterHouse home automation Perl system to Asterisk, the open source telephony system. You can now phone up on the office line and turn the lights/heating/etc. on and off by pressing buttons in response to prompts. It uses asterisk System() calls in the dialplan to run the mhsend script that comes with MisterHouse. I happen to have both systems on the same box, but mhsend works across the network, so you could have it on any networked computer. To give you an example of how it did it, a chunk of my dialplan follows.
Read More
Read More