o Zh@sddlZddlZddlZddlZddlZddlZddlmZmZdda    dddZ dd Z d d Z d d Z ddZddZgadedefddZ   dddZdS)N)profileProfilerActivitycCsdS)NrrrO/var/www/auris/lib/python3.10/site-packages/torch/_functorch/benchmark_utils.py synchronize src  Csr|durdg}|dgkrtjrtjja|duri}|dur!i}|>tdtdD] } ||fi|tq-tdt} t|D] } ||fi|tqHt} Wdn1sdwY| | } tdd|i|4} |"ttdt|D] } ||fi|tqWdn1swYWdn1swY| || S)a0 Output the chrome trace of running f(input, **kwargs_for_f) with [optimize_ctx] [num_runs] times to [trace_filename]. [activities] are the activities that the profiler will record, e.g. ProfilerActivity.CUDA. Return total runtime without the profiler Outputs to trace_filename Ncudacpui9 activitiesr) torchrZ is_availablerZ manual_seedrangetime perf_counterrZexport_chrome_trace)finputZtrace_filename optimize_ctxr num_runsdevicesZ kwargs_for_fZkwargs_for_profiler_t0t1ZtimingZprofrrrdump_chrome_tracesD         rcCst|}t|}|d}|S)NZ traceEvents)openjsonload)filenamerdataeventsrrrget_chrome_trace_eventsKs rcCs(d|vo|dtvod|vo|ddkS)NpidphX)gpu_pidseventrrris_gpu_compute_eventRs  r&cCs4g}|D] }t|s q||qt|tddS)Nts)key)r&appendsortedoperator itemgetter)rsorted_gpu_eventsr%rrrget_sorted_gpu_events\s  r.cCst|dkrdS|d}|d|d}|d}|ddD]}t|d|}|d|d}|t||d}t||}q|S)Nrr'Zdurr)lenmax)r-r%Zcurrent_end_timeZtotal_duration start_timeend_timerrr get_durationes  r3cCs6dd}t|}g}|D] }||sq ||q |S)NcSs8d|vod|dvpd|dvpd|dvpd|dvS)NnameZgemmconvZcutlassZwgradrr$rrris_mm_conv_eventts    z7get_sorted_gpu_mm_conv_events..is_mm_conv_event)r.r))rr6Z gpu_eventsZ sorted_eventsr%rrrget_sorted_gpu_mm_conv_eventsss r7r total_lengthcCst|}ga|D]}d|vrq|ddkr$d|ddvr$t|dq|d}t|}t||}t|}t||}||fS)a Process the chrome traces outputs by the pytorch profiler to compute GPU Utilization and percent of times spent on matmul and convolution Args: filename(str): Name of chrome traces file produced by pytorch profiler total_length(float): total length of the process without profiler in second Return: tuple: (GPU Utilization, percent of time spent on matmul and convolution) r4Zprocess_labelsZGPUargslabelsr g.A)rr#r)r.r3r7)rr8rr%r- utilizationZsorted_gpu_mm_conv_eventsmm_conv_utilizationrrrcompute_utilizations   r=tmp_chrome_tracec Csztj|}|st|td||durt}tj||d}t||||t j g|dgd}t ||\} } | | fS)a Benchmark the GPU Utilization and percent of time spent on matmul and convolution operations of running f(input, **kwargs_for_f) with [optimize_ctx] [num_runs] times. It will produce a chrome trace file in trace_folder/trace_file_name.json Example: ``` def f(a): return a.sum() a = torch.rand(2**20, device="cuda") utilization, mm_conv_utilization = benchmark_utilization(f, a, "tmp", trace_file_name = "tmp_chrome_trace") ``` Args: f: function to benchmark input: input to :attr:`f` trace_folder: name of the folder to store the chrome trace optimize_ctx: the context in which f will run trace_file_name: name of the dumped chrome trace file, default to "tmp_chrome_trace" num_runs: number of times to run f, excluding the warm-up runs, default to 1. Return: tuple: (GPU Utilization, percent of time spent on matmul and convolution) zcreate folder Nz.jsonr)rr) ospathexistsmakedirsprint contextlib nullcontextjoinrrCUDAr=) rrZ trace_folderrZtrace_file_namerZisExistZchrome_trace_file_namer8r;r<rrrbenchmark_utilizations( '   rH)rNNN)Nr>r)rDrr+r?rr Ztorch.profilerrrrrrr&r.r3r7r#strfloatr=rHrrrrs0  :  &