• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

ContentMine / thresher / 44
51%

Build:
DEFAULT BRANCH: master
Ran 07 Sep 2014 11:14PM UTC
Jobs 1
Files 13
Run time 7s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

pending completion
44

Pull #12

travis-ci

Blahah
Complete overhaul of thresher architecture

This is the first, and most major step in a complete overhaul of thresher.
The purpose of this is to support the current and near-future needs of
scraperJSON, based on revisiting the design and incorporating a lot of
user feedback.

Major changes:

- all scraping functionality has been moved to the Scraper class
- the Thresher class now only handles selecting a scraper by URL, and running it
- ScraperBox class holds a collection of scrapers and can match them to URLs
- all logging has been removed and the entire module now operates using events

scraperJSON features implemented:

- elements can be nested (fixes #2 and ContentMine/scraperJSON#3)
- elements can depend on 'following' the captured URLs from other elements (fixes #6)
- URLs are resolved (and all redirects followed) before scraping (fixes #10)
- headless pre-rendering is no longer default (for a massive speed/efficiency increase)
Pull Request #12: Overhaul

276 of 276 new or added lines in 7 files covered. (100.0%)

388 of 470 relevant lines covered (82.55%)

10.15 hits per line

New Missed Lines in Diff

Lines Coverage ∆ File
1
100.0
lib/thresher.js
1
100.0
lib/ticker.js
2
100.0
lib/renderer/basic.js
3
100.0
lib/elementQueue.js
8
100.0
lib/scraper.js
34
100.0
lib/renderer/headless.js

Uncovered Existing Lines

Lines Coverage ∆ File
2
100.0
lib/scraperBox.js
5
100.0
lib/file.js
Jobs
ID Job ID Ran Files Coverage
1 44.1 07 Sep 2014 11:14PM UTC 0
82.55
Travis Job 44.1
Source Files on build 44
Detailed source file information is not available for this build.
  • Back to Repo
  • Travis Build #44
  • Pull Request #12
  • PR Base - master (#40)
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc