27348692714

Committed 11 Jun 2026 01:01PM UTC coverage: 87.569%. First build

Build Type

push

github

Committed by TPGF Roundtrip

Commit Message

Add flow round-trip provenance sourcemap (gated, all testers)

Emit a <flow>.sourcemap.json sidecar tying each generated test element back
to its source: full flow-file lineage (source_stack), node id/file/line, and
per-test TML params captured at render time via the interface. Tester-agnostic
plumbing in sourcemap.rb; hooks in SMT8/SMT7(base)/IGXL renderers. Gated by
site_config.roundtrip_sourcemap (off by default), written to a hidden
.roundtrip/ dir. Purely additive: does not change generated program output.

Coverage Stats

54 of 94 new or added lines in 8 files covered. (57.45%)

13363 of 15260 relevant lines covered (87.57%)

6620.09 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

31.03

/lib/origen_testers/sourcemap.rb

require 'json'

module OrigenTesters
  # Round-trip provenance sidecar (tester-agnostic plumbing).
  #
  # Every tester renderer walks the same ATP AST, and every AST node already carries
  # provenance: a unique id, source file/line, the full flow-file lineage
  # (source_stack), and any domain provenance a plugin attached via options[:meta]
  # (the rt_* keys). This module collects that provenance as each output element is
  # rendered and writes it to a `<output_file>.sourcemap.json` sidecar.
  #
  # It is deliberately platform-neutral: it does NOT know what an SMT8 "suite", an
  # SMT7 test-instance, or an IGXL flow-row is. The renderer is responsible for the
  # one platform-specific thing -- the join KEY (the element's name as it appears in
  # that platform's output file) -- which it passes to record_sourcemap_entry. The
  # collection, meta extraction, append-merge, and file writing are all shared here.
  #
  # To enable for a tester:
  #   include OrigenTesters::Sourcemap
  #   - call record_sourcemap_entry(node, key, kind) from each on_* render hook
  #   - reset_sourcemap before the AST walk (e.g. in finalize)
  #   - call write_sourcemap_file after the output file is written (e.g. write_to_file)
  #
  # Purely additive: it only reads strings already on the AST nodes and writes an
  # extra file. It never changes the generated program output.
  module Sourcemap
    # Append a provenance record for a single rendered output element.
    #
    # @param node [OrigenTesters::ATP::AST::Node] the AST node being rendered
    # @param key [String] the element's name as it appears in the platform output
    #   file (the join key the reverse pass uses to match a diff back to source)
    # @param kind [String] element kind, e.g. 'test', 'sub_flow', 'auxiliary_flow'
    # @param test_method [Object, nil] the TML-aware test method, captured at RENDER
    #   time (the end-state right before the output is written, after all finalize and
    #   any user modification). Its parameter keys are the authoritative native tester
    #   parameter names + final rendered values for the loaded client/TML version.
    # Gate: the sourcemap is OFF by default and only emitted when a site enables it via
    # site_config (AMD turns it on internally; upstream/non-AMD users see no change and
    # pay no cost). Returns false unless Origen.site_config.roundtrip_sourcemap is truthy.
    def roundtrip_sourcemap_enabled?
      return @roundtrip_sourcemap_enabled unless @roundtrip_sourcemap_enabled.nil?
      @roundtrip_sourcemap_enabled = begin
        v = Origen.site_config.respond_to?(:roundtrip_sourcemap) ? Origen.site_config.roundtrip_sourcemap : nil
        [true, 'true', 1, '1'].include?(v)
      rescue StandardError
        false
      end
    end

    def record_sourcemap_entry(node, key, kind, test_method = nil)
      # When disabled, do no work at all -- skip the (potentially expensive) TML
      # format() collection entirely so there is zero render-time overhead.
      return unless roundtrip_sourcemap_enabled?
      @sourcemap ||= []
      entry = {
        # 'suite' retained as the key field name for backward compatibility with the
        # reverse tool; it holds whatever join key the platform supplied.
        'suite'        => key.to_s,
        'kind'         => kind,
        'node_id'      => (node.respond_to?(:id) ? node.id : nil),
        'source_file'  => (node.respond_to?(:file) ? node.file : nil),
        'source_line'  => (node.respond_to?(:line_number) ? node.line_number : nil),
        'source_stack' => (node.respond_to?(:source_stack) ? node.source_stack : nil)
      }
      prov = extract_meta_provenance(node)
      tml = extract_tml_params(test_method)
      entry['provenance'] = prov unless prov.empty?
      entry['tml_params'] = tml unless tml.empty?
      @sourcemap << entry
    end

    # Capture the FINAL native tester parameter names + values for this test, at render
    # time (the true end-state, after build + finalize + any user changes, immediately
    # before the param is written to the output file).
    #
    # The actual extraction is AMD/TML-specific knowledge (what a TML param is, how to
    # read its rendered value), so it is OWNED by amd_test_helpers and delegated to here
    # via the interface: if the loaded interface defines roundtrip_capture_tml_params,
    # we call it. This keeps origen_testers generic (no TML coupling) while still firing
    # at the correct render-time moment, which is the only point the live finalized
    # test_method is guaranteed reachable (SMT8 sub-flows render in forked processes, so
    # a later program-level callback cannot see these objects).
    def extract_tml_params(test_method)
      return {} unless test_method
      iface = (defined?(Origen) && Origen.respond_to?(:interface_loaded?) && Origen.interface_loaded?) ? Origen.interface : nil
      return {} unless iface && iface.respond_to?(:roundtrip_capture_tml_params)
      result = iface.roundtrip_capture_tml_params(test_method)
      result.is_a?(Hash) ? result : {}
    rescue StandardError
      {}
    end

    # Pull the rt_* (and any other) attributes out of the node's (meta ...) child.
    # Structure is: (meta (attribute "rt_burst" "value") (attribute "rt_mode" "x") ...)
    def extract_meta_provenance(node)
      result = {}
      return result unless node.respond_to?(:find)
      meta = node.find(:meta)
      return result unless meta
      meta.to_a.each do |attr|
        next unless attr.respond_to?(:type) && attr.type == :attribute
        k, v = *attr.to_a
        result[k.to_s] = v.to_s unless k.nil?
      end
      result
    end

    def sourcemap_entries
      @sourcemap ||= []
    end

    # Reset the accumulator. Call before each AST walk (finalize may run >once) so
    # re-rendering does not duplicate entries.
    def reset_sourcemap
      @sourcemap = []
    end

    # Where the sidecar is written: a hidden '.roundtrip/' subdirectory NEXT TO the
    # generated output file, named '<output-basename>.sourcemap.json'.
    #
    # The sourcemap is a DERIVED build artifact (regenerated from source every run) and
    # is large, so it must NOT be checked into revision control. Placing it under a
    # dedicated hidden dir keeps it out of the tester-facing output and makes it a single
    # trivially-gitignorable entry ('output/**/.roundtrip/'). The reverse tool reads it
    # from there for the run that produced the program; it is never committed.
    def sourcemap_output_file
      out = output_file
      base = out.basename(out.extname).to_s
      dir = out.dirname.join('.roundtrip')
      FileUtils.mkdir_p(dir.to_s) unless dir.exist?
      Pathname.new(dir.join("#{base}.sourcemap.json"))
    end

    def write_sourcemap_file
      return if sourcemap_entries.empty?
      return unless Origen.interface.respond_to?(:write?) ? Origen.interface.write? : true
      entries = sourcemap_entries
      # When several flow objects append to the same output file, each writes its own
      # portion; merge with whatever is already on disk so the sidecar stays in sync
      # with the appended output content.
      if @append && sourcemap_output_file.exist?
        begin
          existing = JSON.parse(File.read(sourcemap_output_file))
          entries = (existing['entries'] || []) + entries
        rescue StandardError
          # If the existing sidecar is unreadable, fall back to current entries only
        end
      end
      doc = {
        'version'      => 1,
        'flow'         => output_file.basename.to_s,
        'flow_path'    => output_file.to_s,
        'generated_by' => 'origen_testers roundtrip provenance',
        'entries'      => entries
      }
      File.open(sourcemap_output_file, 'w') { |f| f.puts JSON.pretty_generate(doc) }
      Origen.log.info "Writing... #{sourcemap_output_file.basename}"
    end
  end
end

1	require 'json'	4✔
2
3	module OrigenTesters	4✔
4	# Round-trip provenance sidecar (tester-agnostic plumbing).
5	#
6	# Every tester renderer walks the same ATP AST, and every AST node already carries
7	# provenance: a unique id, source file/line, the full flow-file lineage
8	# (source_stack), and any domain provenance a plugin attached via options[:meta]
9	# (the rt_* keys). This module collects that provenance as each output element is
10	# rendered and writes it to a `<output_file>.sourcemap.json` sidecar.
11	#
12	# It is deliberately platform-neutral: it does NOT know what an SMT8 "suite", an
13	# SMT7 test-instance, or an IGXL flow-row is. The renderer is responsible for the
14	# one platform-specific thing -- the join KEY (the element's name as it appears in
15	# that platform's output file) -- which it passes to record_sourcemap_entry. The
16	# collection, meta extraction, append-merge, and file writing are all shared here.
17	#
18	# To enable for a tester:
19	# include OrigenTesters::Sourcemap
20	# - call record_sourcemap_entry(node, key, kind) from each on_* render hook
21	# - reset_sourcemap before the AST walk (e.g. in finalize)
22	# - call write_sourcemap_file after the output file is written (e.g. write_to_file)
23	#
24	# Purely additive: it only reads strings already on the AST nodes and writes an
25	# extra file. It never changes the generated program output.
26	module Sourcemap	4✔
27	# Append a provenance record for a single rendered output element.
28	#
29	# @param node [OrigenTesters::ATP::AST::Node] the AST node being rendered
30	# @param key [String] the element's name as it appears in the platform output
31	# file (the join key the reverse pass uses to match a diff back to source)
32	# @param kind [String] element kind, e.g. 'test', 'sub_flow', 'auxiliary_flow'
33	# @param test_method [Object, nil] the TML-aware test method, captured at RENDER
34	# time (the end-state right before the output is written, after all finalize and
35	# any user modification). Its parameter keys are the authoritative native tester
36	# parameter names + final rendered values for the loaded client/TML version.
37	# Gate: the sourcemap is OFF by default and only emitted when a site enables it via
38	# site_config (AMD turns it on internally; upstream/non-AMD users see no change and
39	# pay no cost). Returns false unless Origen.site_config.roundtrip_sourcemap is truthy.
40	def roundtrip_sourcemap_enabled?	4✔
41	return @roundtrip_sourcemap_enabled unless @roundtrip_sourcemap_enabled.nil?	12,800✔
42	@roundtrip_sourcemap_enabled = begin
43	v = Origen.site_config.respond_to?(:roundtrip_sourcemap) ? Origen.site_config.roundtrip_sourcemap : nil	440✔
44	[true, 'true', 1, '1'].include?(v)	440✔
45	rescue StandardError
NEW 46	false	×
47	end
48	end
49
50	def record_sourcemap_entry(node, key, kind, test_method = nil)	4✔
51	# When disabled, do no work at all -- skip the (potentially expensive) TML
52	# format() collection entirely so there is zero render-time overhead.
53	return unless roundtrip_sourcemap_enabled?	12,800✔
NEW 54	@sourcemap \|\|= []	×
55	entry = {
56	# 'suite' retained as the key field name for backward compatibility with the
57	# reverse tool; it holds whatever join key the platform supplied.
NEW 58	'suite' => key.to_s,	×
59	'kind' => kind,
NEW 60	'node_id' => (node.respond_to?(:id) ? node.id : nil),	×
NEW 61	'source_file' => (node.respond_to?(:file) ? node.file : nil),	×
NEW 62	'source_line' => (node.respond_to?(:line_number) ? node.line_number : nil),	×
NEW 63	'source_stack' => (node.respond_to?(:source_stack) ? node.source_stack : nil)	×
64	}
NEW 65	prov = extract_meta_provenance(node)	×
NEW 66	tml = extract_tml_params(test_method)	×
NEW 67	entry['provenance'] = prov unless prov.empty?	×
NEW 68	entry['tml_params'] = tml unless tml.empty?	×
NEW 69	@sourcemap << entry	×
70	end
71
72	# Capture the FINAL native tester parameter names + values for this test, at render
73	# time (the true end-state, after build + finalize + any user changes, immediately
74	# before the param is written to the output file).
75	#
76	# The actual extraction is AMD/TML-specific knowledge (what a TML param is, how to
77	# read its rendered value), so it is OWNED by amd_test_helpers and delegated to here
78	# via the interface: if the loaded interface defines roundtrip_capture_tml_params,
79	# we call it. This keeps origen_testers generic (no TML coupling) while still firing
80	# at the correct render-time moment, which is the only point the live finalized
81	# test_method is guaranteed reachable (SMT8 sub-flows render in forked processes, so
82	# a later program-level callback cannot see these objects).
83	def extract_tml_params(test_method)	4✔
NEW 84	return {} unless test_method	×
NEW 85	iface = (defined?(Origen) && Origen.respond_to?(:interface_loaded?) && Origen.interface_loaded?) ? Origen.interface : nil	×
NEW 86	return {} unless iface && iface.respond_to?(:roundtrip_capture_tml_params)	×
NEW 87	result = iface.roundtrip_capture_tml_params(test_method)	×
NEW 88	result.is_a?(Hash) ? result : {}	×
89	rescue StandardError
NEW 90	{}	×
91	end
92
93	# Pull the rt_* (and any other) attributes out of the node's (meta ...) child.
94	# Structure is: (meta (attribute "rt_burst" "value") (attribute "rt_mode" "x") ...)
95	def extract_meta_provenance(node)	4✔
NEW 96	result = {}	×
NEW 97	return result unless node.respond_to?(:find)	×
NEW 98	meta = node.find(:meta)	×
NEW 99	return result unless meta	×
NEW 100	meta.to_a.each do \|attr\|	×
NEW 101	next unless attr.respond_to?(:type) && attr.type == :attribute	×
NEW 102	k, v = *attr.to_a	×
NEW 103	result[k.to_s] = v.to_s unless k.nil?	×
104	end
NEW 105	result	×
106	end
107
108	def sourcemap_entries	4✔
109	@sourcemap \|\|= []	760✔
110	end
111
112	# Reset the accumulator. Call before each AST walk (finalize may run >once) so
113	# re-rendering does not duplicate entries.
114	def reset_sourcemap	4✔
115	@sourcemap = []	972✔
116	end
117
118	# Where the sidecar is written: a hidden '.roundtrip/' subdirectory NEXT TO the
119	# generated output file, named '<output-basename>.sourcemap.json'.
120	#
121	# The sourcemap is a DERIVED build artifact (regenerated from source every run) and
122	# is large, so it must NOT be checked into revision control. Placing it under a
123	# dedicated hidden dir keeps it out of the tester-facing output and makes it a single
124	# trivially-gitignorable entry ('output/**/.roundtrip/'). The reverse tool reads it
125	# from there for the run that produced the program; it is never committed.
126	def sourcemap_output_file	4✔
NEW 127	out = output_file	×
NEW 128	base = out.basename(out.extname).to_s	×
NEW 129	dir = out.dirname.join('.roundtrip')	×
NEW 130	FileUtils.mkdir_p(dir.to_s) unless dir.exist?	×
NEW 131	Pathname.new(dir.join("#{base}.sourcemap.json"))	×
132	end
133
134	def write_sourcemap_file	4✔
135	return if sourcemap_entries.empty?	760✔
NEW 136	return unless Origen.interface.respond_to?(:write?) ? Origen.interface.write? : true	×
NEW 137	entries = sourcemap_entries	×
138	# When several flow objects append to the same output file, each writes its own
139	# portion; merge with whatever is already on disk so the sidecar stays in sync
140	# with the appended output content.
NEW 141	if @append && sourcemap_output_file.exist?	×
142	begin
NEW 143	existing = JSON.parse(File.read(sourcemap_output_file))	×
NEW 144	entries = (existing['entries'] \|\| []) + entries	×
145	rescue StandardError
146	# If the existing sidecar is unreadable, fall back to current entries only
147	end
148	end
149	doc = {
NEW 150	'version' => 1,	×
151	'flow' => output_file.basename.to_s,
152	'flow_path' => output_file.to_s,
153	'generated_by' => 'origen_testers roundtrip provenance',
154	'entries' => entries
155	}
NEW 156	File.open(sourcemap_output_file, 'w') { \|f\| f.puts JSON.pretty_generate(doc) }	×
NEW 157	Origen.log.info "Writing... #{sourcemap_output_file.basename}"	×
158	end
159	end
160	end

Origen-SDK / origen_testers / 27348692714

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous