python web scraping an unstructured table

You can use beautifulsoup to parse the HTML. For example:

import pandas as pd
from bs4 import BeautifulSoup

txt = '''<table class="table-detail">
                    <td colspan="4" class="noborder">General Information
                    <th>Full name</th>
                        James Smith
                    <th>Year of birth</th>
                    <th>Place of birth</th>
                    <td>TTexas, USA</td>
                    <td>Texas, USA</td>

soup = BeautifulSoup(txt, 'html.parser')

row = {}
for h in'th:has(+td)'):
    row[h.text] = h.find_next('td').get_text(strip=True)

df = pd.DataFrame([row])


     Full name Year of birth Gender Place of birth     Address
0  James Smith          1992   Male    TTexas, USA  Texas, USA

CLICK HERE to find out more related problems solutions.

Leave a Comment

Your email address will not be published.

Scroll to Top