The Movie Script Scraper is a simple Node.js based web scraper to retrieve movie scripts by genre or title from IMSDB.
Installation
npm install -S movie-script-scraper
Usage
Example Usage
Movie Script Scraper exposes a function; simply pass this function the options (see below), and it will return a promise with an array of the file paths of the scripts it saved.const mss = require('movie-script-scraper');
const options = {
genre: 'Action',
total: 10,
};
mss(options)
.then(filePaths => {
console.log(filePaths);
})
.catch(err => {
console.error('There was a problem');
});
Options
genre[string] – Any valid film genre, a complete list can be found here.- Defaults to “Action”.
total[number] – the total number of scripts you want from a given genre.- Defaults to 10.
title[string] – The name of the film’s script you want.dest[string] – Location that you want to save your scripts.- Defaults to ./scripts in the root directory.
Running from command line
You can run the Movie Script Scraper directly from the CLI (if it’s globally available in your PATH, e.g. bynpm install -g movie-script-scraper) with a variety of useful options.
movie-script-scraper --total 10 --genre Comedy
movie-script-scraper --title 'american sniper'
How it Works
Conveniently IMSDB provides RSS feeds based on movie genre (ex. http://www.imsdb.com/feeds/genre.php?genre=Comedy). Using the awesome fetch module we are then able to grab that page and use a regular expression to generate an array of movie script URLs. We then visit each URL with Fetch and use another awesome module called Cheerio to select just the movie script from each page and finally output each script to a file.Running Locally
- Install dependencies with:
npm install
- Run Tests
npm test
Contributing
Don’t hesitate to create a pull request. Every contribution is appreciated. In development, you can start the tests by callingnpm test. Check out our contribution README for more info. TLDR;
- Fork it!
- Create your feature branch:
git checkout -b my-new-feature - Commit your changes:
git commit -am 'Add some feature' - Push to the branch:
git push origin my-new-feature - Submit a pull request 😀
Maintainers
 Joe Karlsson |
License
MIT
Follow Joe Karlsson on Social
- Twitter – https://twitter.com/JoeKarlsson1
- TikTok – https://www.tiktok.com/@joekarlsson
- GitHub – https://github.com/JoeKarlsson
- YouTube – https://www.youtube.com/c/JoeKarlsson
- Twitch – https://www.twitch.tv/joe_karlsson
- Medium – https://medium.com/@joekarlsson
- LinkedIn – https://www.linkedin.com/in/joekarlsson/
- Reddit – www.reddit.com/user/joekarlsson
- Instagram – https://www.instagram.com/joekarlsson/
Want to Learn More About Joe Karlsson?