Structured extraction selects useful fields from a web page or document and reshapes them into data ready for a custom visualization. It replaces tedious manual copying with either a quick table-conversion service or a script when finer control matters. section 14.1
A practical extraction workflow:
- Map source structures to output categories, such as assigning painting-table indexes to oil, watercolor, or print mediums. section 14.1
- Select matching HTML tables with document.querySelectorAll(), then iterate through their rows. section 14.1
- Collect only the fields needed for the visualization, including each painting’s title, image URL, and medium. section 14.1
Van Gogh painting data
A Wikipedia page contains more than a thousand paintings across HTML tables. A browser-console script extracts selected paintings for a timeline of Van Gogh’s final ten years while leaving letter sketches aside. section 14.1
Listing 14.1 Extracting data from Van Gogh’s paintings on Wikipedia
const tables = [ #1
{ index: 0, medium: "oil" },
{ index: 1, medium: "oil" },
{ index: 2, medium: "oil" },
{ index: 3, medium: "oil" },
{ index: 4, medium: "oil" },
{ index: 5, medium: "oil" },
{ index: 6, medium: "watercolor" },
{ index: 7, medium: "print" },
{ index: 8, medium: "print" },
];
const domTables = document.querySelectorAll( #2
".wikitable.sortable"
);
const paintings = []; #3
domTables.forEach((table, i) => { #4
if (i <= tables.length - 1) {
const medium = tables[i].medium;
const rows = table.querySelectorAll("tbody tr");
rows.forEach(row => {
let title; #5
if (row.querySelector(".thumbcaption i a")) {
title = row.querySelector(".thumbcaption i a")
.textContent;
} else if (row.querySelector(".thumbcaption i")) {
title = row.querySelector(".thumbcaption i")
.textContent;
} else {
title = null;
}
let imageLink; #6
if (row.querySelector(".thumbinner img")) {
const image = row.querySelector(
".thumbinner img"
).srcset;
imageLink = `https${image.slice(
image.indexOf("1.5x, ")+6,-3
)}`;
} else {
imageLink = null;
}
paintings.push({ #7
title: title,
imageLink: imageLink,
medium: medium
});
})
}
});
console.log(paintings); #8
Use the complete listing to see table classification, DOM selection, and the start of the paintings collection implemented in JavaScript. section 14.1
