fTh%SSKJrJrJr SSKJr SSKJrJrJ r J r SSK J r J r \ "5(a SSKJr SSKJr \"5(a SS KJr SS KJr \ R,"\5r\"\ "S S S 95"S S\ 55rg))ListOptionalUnion)GenerationConfig)add_end_docstringsis_torch_availableis_vision_availablelogging)Pipelinebuild_pipeline_init_args)Image) load_image)1MODEL_FOR_VISUAL_QUESTION_ANSWERING_MAPPING_NAMES) KeyDatasetT) has_tokenizerhas_image_processorc ^\rSrSrSrSr\"SS9rU4SjrSSjr SS\ S \ \ S \ \ S 4S \ \ \ \ \ 44U4S jjjrSS jrSrSSjrSrU=r$)VisualQuestionAnsweringPipelineaS Visual Question Answering pipeline using a `AutoModelForVisualQuestionAnswering`. This pipeline is currently only available in PyTorch. Unless the model you're using explicitly sets these generation parameters in its configuration files (`generation_config.json`), the following default values will be used: - max_new_tokens: 256 Example: ```python >>> from transformers import pipeline >>> oracle = pipeline(model="dandelin/vilt-b32-finetuned-vqa") >>> image_url = "https://huggingface.co/datasets/Narsil/image_dummy/raw/main/lena.png" >>> oracle(question="What is she wearing ?", image=image_url) [{'score': 0.948, 'answer': 'hat'}, {'score': 0.009, 'answer': 'fedora'}, {'score': 0.003, 'answer': 'clothes'}, {'score': 0.003, 'answer': 'sun hat'}, {'score': 0.002, 'answer': 'nothing'}] >>> oracle(question="What is she wearing ?", image=image_url, top_k=1) [{'score': 0.948, 'answer': 'hat'}] >>> oracle(question="Is this a person ?", image=image_url, top_k=1) [{'score': 0.993, 'answer': 'yes'}] >>> oracle(question="Is this a man ?", image=image_url, top_k=1) [{'score': 0.996, 'answer': 'no'}] ``` Learn more about the basics of using a pipeline in the [pipeline tutorial](../pipeline_tutorial) This visual question answering pipeline can currently be loaded from [`pipeline`] using the following task identifiers: `"visual-question-answering", "vqa"`. The models that this pipeline can use are models that have been fine-tuned on a visual question answering task. See the up-to-date list of available models on [huggingface.co/models](https://huggingface.co/models?filter=visual-question-answering). T)max_new_tokenscP>[TU]"U0UD6 UR[5 gN)super__init__check_model_typer)selfargskwargs __class__s h/var/www/auris/envauris/lib/python3.13/site-packages/transformers/pipelines/visual_question_answering.pyr(VisualQuestionAnsweringPipeline.__init__Bs$ $)&) OPc 00pvUbX&S'UbX6S'UbXFS'UbXS'0n[USS5bURUS'[USS5bURUS'URUS'XhU4$)Npadding truncationtimeouttop_kassistant_modelassistant_tokenizer tokenizer)getattrr+r-r,) rr*r'r(r)r!preprocess_paramspostprocess_paramsforward_paramss r#_sanitize_parameters4VisualQuestionAnsweringPipeline._sanitize_parametersFs02B-  +2i (  !.8l +  +2i (  */w ' 4*D 1 =040D0DN, - 4. 5 A*...N; '484L4LN0 1 2DDDr%imagez Image.Imagerquestionc >[U[5n[U[5=(a [SU55n[U[5=(a [SU55n[U[R[ 45(a[U[ 5(aXS.nOU(dU(a*[U[ 5(aUVs/sHoUS.PM nnO[U[R[ 45(aU(aUV s/sHoU S.PM nn OBU(dU(a1U(a*/n UHn UHnU R XS.5 M M! U nOUn[T U] "U40UD6n U $s snfs sn f)a Answers open-ended questions about images. The pipeline accepts several types of inputs which are detailed below: - `pipeline(image=image, question=question)` - `pipeline({"image": image, "question": question})` - `pipeline([{"image": image, "question": question}])` - `pipeline([{"image": image, "question": question}, {"image": image, "question": question}])` Args: image (`str`, `List[str]`, `PIL.Image`, `List[PIL.Image]` or `KeyDataset`): The pipeline handles three types of images: - A string containing a http link pointing to an image - A string containing a local path to an image - An image loaded in PIL directly The pipeline accepts either a single image or a batch of images. If given a single image, it can be broadcasted to multiple questions. For dataset: the passed in dataset must be of type `transformers.pipelines.pt_utils.KeyDataset` Example: ```python >>> from transformers.pipelines.pt_utils import KeyDataset >>> from datasets import load_dataset >>> dataset = load_dataset("detection-datasets/coco") >>> oracle(image=KeyDataset(dataset, "image"), question="What's in this image?") ``` question (`str`, `List[str]`): The question(s) asked. If given a single question, it can be broadcasted to multiple images. If multiple images and questions are given, each and every question will be broadcasted to all images (same effect as a Cartesian product) top_k (`int`, *optional*, defaults to 5): The number of top labels that will be returned by the pipeline. If the provided number is higher than the number of labels available in the model configuration, it will default to the number of labels. timeout (`float`, *optional*, defaults to None): The maximum time in seconds to wait for fetching images from the web. If None, no timeout is set and the call may block forever. Return: A dictionary or a list of dictionaries containing the result. The dictionaries contain the following keys: - **label** (`str`) -- The label identified by the model. - **score** (`int`) -- The score attributed by the model for that label. c3b# UH%n[U[R[45v M' g7fr) isinstancerstr.0items r# ;VisualQuestionAnsweringPipeline.__call__..s'8pjobfD5;;X[J\9]9]jos-/c3B# UHn[U[5v M g7fr)r8r9r:s r#r=r>s>jaiY]z$PS?T?Tais)r4r5) r8rlistallrr9appendr__call__) rr4r5r! is_datasetis_image_batchis_question_batchinputsimqquestion_image_pairsresultsr"s r#rC(VisualQuestionAnsweringPipeline.__call__Zs;f z2 #E40pS8pjo8p5p&x6j3>jai>j;j eekk3/ 0 0Z#5N5N$;F 8S0I0IDIJEb9EFJF  S1 2 27H?GHx!15xFHF 0A#% B(//"0LM *F  F'"64V4%KHs E1 E6c[USUS9nURUSURUUS9nURXPRS9nURS:XaUR UR 5nUR U5 U$)Nr4)r)r5)return_tensorsr'r()imagesrNpt)rr- frameworkimage_processorto torch_dtypeupdate)rrGr'r(r)r4 model_inputsimage_featuress r# preprocess*VisualQuestionAnsweringPipeline.preprocesss6'?G<~~ : >>! & --U>>-Z >>T !+..t/?/?@NN+r%c URR5(a5SU;aURUS'URR"S0UDUD6nU$UR"S0UD6nU$)Ngeneration_config)model can_generater[generate)rrVgenerate_kwargs model_outputss r#_forward(VisualQuestionAnsweringPipeline._forwardsk :: " " $ $"/97;7M7M 34 JJ//R,R/RM!JJ66Mr%c URR5(a;UVs/sH-nSURRUSS9R 50PM/ sn$X RR R :a URR R nURS:Xa1URR5SnURU5upVO[SUR35eUR5nUR5n[XV5VVs/sH*upxXpRR RUS.PM, snn$s snfs snnf)NanswerT)skip_special_tokensrPrzUnsupported framework: )scorere)r]r^r-decodestripconfig num_labelsrQlogitssigmoidtopk ValueErrortolistzipid2label) rrar* output_idsprobsscoresidsrg_ids r# postprocess+VisualQuestionAnsweringPipeline.postprocesss0 :: " " $ $#0"/J4>>00QU0V\\^_"/  zz((333 ))44~~%%,,446q9#jj/  #:4>>:J!KLL]]_F**,CbeflbrsbrT^TYezz/@/@/I/I#/NObrs s! ts 4E%1Er\)NNNNr)FFN))__name__ __module__ __qualname____firstlineno____doc___pipeline_calls_generater_default_generation_configrr2rr9rrrCrXrbrx__static_attributes__ __classcell__)r"s@r#rrs$L $!1"QE.59L]Cm)rsw(()XX4(]$   H %,4UYZ[~th~t\~tr%