|
|
|
|
|
Grabbing sites
Let us have a look at the web page you have in front of you right now. This page contains some text, a few images and links to other pages, in this case only links to pages on the same site, www.zultrax.com, but it could also contain links to other sites.
The principle of grabbing is to click on each link and see what comes. Does the page that comes contain images than we save them, and we click also on each link on that page.
Pretty much boring thing to do, but hey, that's why you bought a computer, it can do work for you. For the computer it ain't boring, it is what computers are good in, doing a lot of work, very fast.
The page where you are looking at now, it is a rather common page, contains a little more than 20 links to other pages. Once the page is processed GrabJPG has 20 new tasks to perform. Those tasks are not performed sequential but - within the limits of the available resourses - simultaneously. Already one of the reasons the program can perform such an action much more fast than a human ever could.
Of course the program also takes note of which pages have already been done. So if another page links back to this page, it won't process it twice. It also won't find you the same image twice. Here the actually found image is compared with images found before. If it is just a copy of something we already found, it is ignored.
That simple principle will have a site like this one scanned within a few minutes. Larger sites will take longer.
Limiting it
Grabbing a website can go very fast. That is because we made one implicit condition: We won't follow links that link to other sites. If we would do that, we would end up in scanning the entire internet. Our site links to about 40 other sites, some of them to hundreds of others, and so on.
GrabJPG can also lift this limitation but you'll need to put another one instead, avoiding a useless scan of thousands of webpages. Take for example directory.yahoo.com, a site full with categorized links to other sites. If you find the right category and start grabbing from there, you'll find a lot of images related to that category. Nice, but put 2 limitations:
- Don't scan pages on directory.yahoo.com itself, since, that would include also the enourmous yahoo site in your grab. It can take a few months.
- Set a maximum link depth. If you set this to e.g. 3 it follows all links on the page, all links on the page the page links to and one level deeper but all the rest will be ignored. After a few levels the chance that you still sit on something related to the subject is anyway small.
Make it smart
Getting the right images means pretty much grabbing the right site. But getting them good and fast means using the right software.
GrabJPG has a lot of built in knowledge and tricks to avoid overhead and scan a site smart.
We scan only for JPG files, so we ignore another popular image format, GIF. Like around 99% of all GIF images are images used for making the lay out of a site or for advertisement banners. The rest might be interesting, nice pictures, you would want to see. By skipping them we skip also 99% of the crap, we won't have t read it.
Another example. A site contains a folder named FORUM and a folder named PICTURES. GrabJPG looks with much more attention to the second one than to the first one.
The last example we want to give is in the security. Websites can contain all kind of doubfull content stuff as scripts and ActiveX. They are the cause of evil, the famous Microsoft Browser leaks. GrabJPG scans sites, not visits them and surely not allows them to run any scripts on your computer. Who, besides Microsoft, would find it a good idea that www.sexypictures.ru can run scripts on your computer, no matter how good it is protected. The best protection is, nothing gets opened, nothing gets displayed, nothing can run. We scan the site and that is it. Sometimes we miss a few pictures but it is the only way to stay 100% safe.
With security we are finished. Anything that could potentially harm will be fully kept out. On the smart grabbing we keep on working, avoiding as much overhead as possible, and getting you your images as fast as possible.
|
|
|
|
|
|
