I'm trying to scrape Wikipedia as a project to learn a bit of python 3. I've managed to get the links from a page:
import urllib.request from bs4 import BeautifulSoup html_code = urllib.request.urlopen('https://en.wikipedia.org/wiki/Category:Lists_of_airports_by_country').read().decode() souped_code = BeautifulSoup(html_code, "html.parser") for element in souped_code.find_all("a"): dest_links = element.get('href') print(dest_links)
but I just get a series of strings I'd like to work with (such as in a list so can use indexes and only keep the "List_of_airports_in_" links) and filter, open, iterate over, etc. them, but I just can't wrap my head around how to achieve this as it seems to produce a series of strings.
Any insights will be much appreciated!