Commit 0375c800 authored by Jay Young's avatar Jay Young Committed by GitHub
Browse files

Fix `Segmentation Fault` when loading very big corpus file

example_pos_ is the cursor to current position when reading the corpus file. When corpus file is very big, the corpus_size_ may bigger than max of int32, then example_pos_ became negative, and cause `corpus_[example_pos_]` to `Segmentation Fault`.
parent 3bf85a4e
......@@ -128,7 +128,7 @@ class SkipgramWord2vecOp : public OpKernel {
random::SimplePhilox rng_ GUARDED_BY(mu_);
int32 current_epoch_ GUARDED_BY(mu_) = -1;
int64 total_words_processed_ GUARDED_BY(mu_) = 0;
int32 example_pos_ GUARDED_BY(mu_);
int64 example_pos_ GUARDED_BY(mu_);
int32 label_pos_ GUARDED_BY(mu_);
int32 label_limit_ GUARDED_BY(mu_);
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment