Does anyone have any experience with using Webscraper.Io to scrape websites for data? I have no coding experience so I am at the mercy of plugins, extensions and tools. I am working to build a daily dashboard of newly released government contract opportunities. I have successfully built the site maps for each of the 20 sites we monitor, but each of the sites exports data to its own Google Sheet.

Does anyone know how I can have each of these sites populate into one google document but with its own tab (instead of its own sheet).

It also looks like I can export data to S3 and can export as a .csv, JSON, or .xlsx.

Any help or guidance would be greatly appreciated.

2 ...

Report Post

Posted by Big_Sur

Member since Nov 2012

1165 posts

Posted on 8/18/24 at 9:11 pm to Drunken Crawfish

Sort of depends what you are trying to pull off exactly.

You want a dashboard for you and a couple of others showing just recent data? Or a website that includes historical data?

For the first you might get by with some Excel macros or something. For the second you probably want to connect to their API with a MySQL or similar database.

1 ...

Report Post

Posted by Drunken Crawfish

Member since Apr 2017

3857 posts

Posted on 8/18/24 at 9:49 pm to Big_Sur

Historical and searchable would be ideal. But there are going to be limits to my capacity. It’ll all be for internal use for our team.

This post was edited on 8/18/24 at 9:57 pm

0 ...

Report Post

Posted by TigerinATL

Member since Feb 2005

62431 posts

Posted on 8/19/24 at 8:41 am to Drunken Crawfish

If you already have the data in separate google sheets and just want to consolidate it into one, importrange lets you pull in data from other google sheets. That's a built in function, here's an article that lists some other options too like extensions. LINK /

This post was edited on 8/19/24 at 8:43 am

Page 1