"...text-generation-inference.git" did not exist on "bc5c07231ec6284b94a8ff78a54d6da4b505a93f"
Commit 0b5def52 authored by Jeremy Reizenstein's avatar Jeremy Reizenstein Committed by Facebook GitHub Bot
Browse files

avoid numpy warning in split

Summary:
avoid creating a numpy array of random things just to split it: this can now generate a warning e.g. if the list contains lists of varying lengths. There might also be a performance win here, and we could do more of the same if we care about that.

(The vanilla way to avoid the new warning is to replace `np.split(a,` with `np.split(np.array(a, dtype=object), ` btw.)

Reviewed By: shapovalov

Differential Revision: D40209308

fbshipit-source-id: daae33a23ceb444e8e7241f72ce1525593e2f239
parent 56d3465b
...@@ -225,8 +225,8 @@ def _dataclass_list_from_dict_list(dlist, typeannot): ...@@ -225,8 +225,8 @@ def _dataclass_list_from_dict_list(dlist, typeannot):
assert indices[-1] == len(all_keys_res) assert indices[-1] == len(all_keys_res)
keys = np.split(list(all_keys_res), indices[:-1]) keys = np.split(list(all_keys_res), indices[:-1])
vals = np.split(list(all_vals_res), indices[:-1]) all_vals_res_iter = iter(all_vals_res)
return [cls(zip(k, v)) for k, v in zip(keys, vals)] return [cls(zip(k, all_vals_res_iter)) for k in keys]
elif not dataclasses.is_dataclass(typeannot): elif not dataclasses.is_dataclass(typeannot):
return dlist return dlist
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment