As the title states, I am trying to get all of the text data from multiple websites. I am able to grab the text data (
<p>) if I use the following:
url = “https://Somerandomwebsite.com” response = requests.get(url, timeout=5) tree = lxml.html.fromstring(response.text) things = tree.find_class(“the class that contains <p>”) data = [_.text_content() for _ in things] Print(data)
However, this is only then for one website, as it grabs the above class and then grabs the text. I would prefer not to go into each website and find the class that the text data is apart of. Is there any way to just search all the websites for the text(
<p>) and then grab all of that data?
Any help would be greatly appreciated.