Best way of scraping pic galeries from any url

General discussions related to php

Moderators: egami, macek, gesf

Post Reply
cali89
New php-forum User
New php-forum User
Posts: 7
Joined: Sun Sep 02, 2018 10:33 am

Sun Sep 02, 2018 10:40 am

Would you have an idea what could be the best and the most universal way of scraping pic galleries from any url (all pic files from any url with these; maybe even omit other not needed pics like ad banners or some other things). I know that when people attempt to write such code it does not necessarily work on all the sites as the format of them differ (like html wise or css wise, maybe javaScript wise too).

I was thinking about searching for the actual .jpg, .jpeg, .png, .gif files first, figuring out the path and then dowloading them. This would be domain based, it could be assumed that the whole domain is set up in the same way (if there would be issues a user could come back to this and review, but this would be getting done).

Basically the task - create the most successful pic gallery scraper on the Net as of 2018. World competition for this, something better that what already can exist (lets say).

What could be the best way of getting this done? Any sources on the Net that I could look at would be good too.

Thanks.

chorn
php-forum Fan User
php-forum Fan User
Posts: 563
Joined: Fri Apr 01, 2016 2:18 am

Mon Sep 03, 2018 3:43 am

use WGET.

Post Reply