ponder-lab / Hybridize-Functions-Refactoring / #3584

89%

main: 90%

push

github

Refuse a trace-time call to the builtin open as a Python side effect (#1090)

* Refuse Python I/O, clock reads, and Python-level randomness that tracing runs once

A function whose body, directly or through the program's own code it
calls, does Python I/O, reads the clock, or draws Python-level
randomness runs that code once under tf.function, while tracing, so its
effect and result are fixed on the first call. HAS_TRACE_TIME_PYTHON_EFFECTS
(37) refuses converting such a function.

Measured on TensorFlow 2.9.3, eagerly and under tf.function, with the
fixture distilling the five functions #1077 cites: both OpenNMT-tf
get_dataset_size overrides, counting lines through tf.io.gfile.GFile,
return the first count again after the file grows; Model.visualize
writes its projector file only on the first call; FCN's
process_image_label and NLPGNN's TUDataset.load return the same random
augmentation and split on every call; and predict_dataset raises
AttributeError.

The calls recognized are the builtin open, read from a script's scope
that does not rebind it; the time module's clock reads; any function of
random or numpy.random; cv2.imread, cv2.imreadmulti, and cv2.imwrite; and
any function of tf.io.gfile and its compatibility spellings. A call is
recognized by the module its attribute chain roots at, by points-to or
by an import alias read from the script's scope, never by a bare member
name, so tf.random.uniform, tf.print, and a time() method on the
program's own object are left alone. The walk mirrors the eager-only-call
analysis, and an alwaysCheckTraceTimePythonEffects knob mirrors
alwaysCheckEagerOnlyCalls.

predict_dataset's numpy() call inside a lambda passed to
tf.nest.map_structure is still unseen by the eager-only check
(wala/ML#1067); it is pinned with a TODO and refused here for its file
I/O and clock reads.

testNumpyOverWildDimension's wild_random_size draws from np.random at
trace time and is now refused with code 37 rather than converted with
it... (continued)

9 of 9 new or added lines in 2 files covered. (100.0%)

5297 of 5975 relevant lines covered (88.65%)

0.89 hits per line