14 Eylül 2012 Cuma

Downloading MIT OCW (Open Course Ware) Course Materials

I want to download some course materials from MIT OCW, but I need to make much of single clicks to download large number of materials. Instead, I followed the procedure below:

1. Open the course pages which are subject of interest in the Chrome (in separete tabs).

2. Save the session using FreshStart Chrome extension.



3. Export the session just saved.


4. FreshStart will give you a JSON like below:


5. Now, we will generate actual download links from home page URL's. Paste the JSON string to your favorite regex editor. I am using RegexBuddy, and strongly recommend it.

6. Search for this regex (newlines added to adjust layout, delete them when using):
"title":"(?:MIT OpenCourseWare \| ([^\|]+) \| (\d+)\.\S+ ([^,]+),[^"]+|[^"]+\bResource\b
[^"]+)","url":"http://ocw\.mit\.edu/(?:courses/([^/]+)/|resources/res-)(\d+-\w+)-([^/]+)
-(spring|fall|\w+-iap)-(\d+)/

7.  Use following replacement regex to generate links for courses:
http://ocw.mit.edu/ans15436/ZipForEndUsers/\2/\5-\7-\8/\5-\7-\8.zip

8. Use following replacement regex to generate links for resources:
http://ocw.mit.edu/ans15436/ZipForEndUsers/RES/res-\5-\7-\8/res-\5-\7-\8.zip

9. Simply copy both course and resource links and JDownloader automatically grab them from your clipboard. Then click "Continue with all" to start downloads.

10. After dowloads completed, you will obtain files that whose names consists of course code, semester and year:
6-046j-fall-2005.zip

You probably want to know human readable couse name, right? This was the main reason to use FreashStart. In this way, we obtained actual course names from page titles and will rename files with them:
6-046j-fall-2005 - Introduction to Algorithms (SMA 5503).zip

Use following replacement regex to generate proper renaming commands for courses:
ren \5-\7-\8.zip "\5-\7-\8 - \3.zip"

11. Unfortunately, we can not obtaine course name from the resource page titles like we do with courses:
Free Online Course Materials | Resource Home | MIT OpenCourseWare

But we can still obtain them from links, but in lowercase and dashes instead of spaces.
res-6-004-spring-2009 - principles-of-computer-system-design-an-introduction.zip

Use following replacement regex to generate proper renaming commands for resources:
ren res-\5-\7-\8.zip "res-\5-\7-\8 - \6.zip"

12. Copy all the generated rename commands and save as filename.bat. Attention! Replace illegal characters (like colon ':') in file names, otherwise they will be skipped.

13. After downloads completed, put all downloaded files and the filename.bat into the same folder and then double-click to  filename.bat.

HTH someone else..

Hiç yorum yok:

WSO2 ESB + ClientAccesPolicy.xml + CrossDomain.xml

ClientAccesPolicy.xml and CrossDomain.xml files have to be found at root of your services Silverlight and Flex clients to be able to acce...