3.3 KiB
This model was released on 2021-10-16 and added to Hugging Face Transformers on 2022-09-30 and contributed by nielsr.
MarkupLM
MarkupLM addresses Visually-rich Document Understanding (VrDU) for digital documents with dynamic layouts, such as HTML/XML-based web pages. By jointly pre-training on text and markup information, MarkupLM enhances document understanding tasks. It achieves state-of-the-art results on WebSRC and SWDE benchmarks, demonstrating superior performance compared to existing models.
import torch
from transformers import AutoProcessor, AutoModelForQuestionAnswering
processor = AutoProcessor.from_pretrained("microsoft/markuplm-base-finetuned-websrc")
model = AutoModelForQuestionAnswering.from_pretrained("microsoft/markuplm-base-finetuned-websrc", dtype="auto")
html_string = "<html> <head> <title>My name is Niels</title> </head> </html>"
question = "What's his name?"
encoding = processor(html_string, questions=question, return_tensors="pt")
with torch.no_grad():
outputs = model(**encoding)
answer_start_index = outputs.start_logits.argmax()
answer_end_index = outputs.end_logits.argmax()
predict_answer_tokens = encoding.input_ids[0, answer_start_index : answer_end_index + 1]
processor.decode(predict_answer_tokens).strip()
Usage tips
- In addition to
input_ids, [~MarkupLMModel.forward] expects 2 additional inputs:xpath_tags_seqandxpath_subs_seq. These are the XPATH tags and subscripts respectively for each token in the input sequence. - Use [
MarkupLMProcessor] to prepare all data for the model. Refer to the usage guide for more information.
MarkupLMConfig
autodoc MarkupLMConfig - all
MarkupLMFeatureExtractor
autodoc MarkupLMFeatureExtractor - call
MarkupLMTokenizer
autodoc MarkupLMTokenizer - build_inputs_with_special_tokens - get_special_tokens_mask - create_token_type_ids_from_sequences - save_vocabulary
MarkupLMTokenizerFast
autodoc MarkupLMTokenizerFast - all
MarkupLMProcessor
autodoc MarkupLMProcessor - call
MarkupLMModel
autodoc MarkupLMModel - forward
MarkupLMForSequenceClassification
autodoc MarkupLMForSequenceClassification - forward
MarkupLMForTokenClassification
autodoc MarkupLMForTokenClassification - forward
MarkupLMForQuestionAnswering
autodoc MarkupLMForQuestionAnswering - forward