Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
OpenDAS
nni
Commits
139e0a90
Unverified
Commit
139e0a90
authored
Jun 03, 2019
by
SparkSnail
Committed by
GitHub
Jun 03, 2019
Browse files
Fix remote gpu scheduler bug (#1143)
parent
ce274f0a
Changes
2
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
7 additions
and
19 deletions
+7
-19
src/nni_manager/training_service/remote_machine/remoteMachineData.ts
...ager/training_service/remote_machine/remoteMachineData.ts
+6
-19
src/nni_manager/training_service/remote_machine/remoteMachineTrainingService.ts
...ng_service/remote_machine/remoteMachineTrainingService.ts
+1
-0
No files found.
src/nni_manager/training_service/remote_machine/remoteMachineData.ts
View file @
139e0a90
...
@@ -29,31 +29,18 @@ import { GPUSummary, GPUInfo } from '../common/gpuData';
...
@@ -29,31 +29,18 @@ import { GPUSummary, GPUInfo } from '../common/gpuData';
* Metadata of remote machine for configuration and statuc query
* Metadata of remote machine for configuration and statuc query
*/
*/
export
class
RemoteMachineMeta
{
export
class
RemoteMachineMeta
{
public
readonly
ip
:
string
;
public
readonly
ip
:
string
=
''
;
public
readonly
port
:
number
;
public
readonly
port
:
number
=
22
;
public
readonly
username
:
string
;
public
readonly
username
:
string
=
''
;
public
readonly
passwd
?
:
string
;
public
readonly
passwd
:
string
=
''
;
public
readonly
sshKeyPath
?:
string
;
public
readonly
sshKeyPath
?:
string
;
public
readonly
passphrase
?:
string
;
public
readonly
passphrase
?:
string
;
public
gpuSummary
:
GPUSummary
|
undefined
;
public
gpuSummary
:
GPUSummary
|
undefined
;
public
readonly
gpuIndices
?:
string
;
public
readonly
gpuIndices
?:
string
;
public
readonly
maxTrialNumPerGpu
?:
number
;
public
readonly
maxTrialNumPerGpu
?:
number
;
public
occupiedGpuIndexMap
:
Map
<
number
,
number
>
;
//TODO: initialize varialbe in constructor
public
occupiedGpuIndexMap
?:
Map
<
number
,
number
>
;
public
readonly
useActiveGpu
?:
boolean
=
false
;
public
readonly
useActiveGpu
?:
boolean
=
false
;
constructor
(
ip
:
string
,
port
:
number
,
username
:
string
,
passwd
:
string
,
sshKeyPath
:
string
,
passphrase
:
string
,
gpuIndices
?:
string
,
maxTrialNumPerGpu
?:
number
,
useActiveGpu
?:
boolean
)
{
this
.
ip
=
ip
;
this
.
port
=
port
;
this
.
username
=
username
;
this
.
passwd
=
passwd
;
this
.
sshKeyPath
=
sshKeyPath
;
this
.
passphrase
=
passphrase
;
this
.
gpuIndices
=
gpuIndices
;
this
.
maxTrialNumPerGpu
=
maxTrialNumPerGpu
;
this
.
occupiedGpuIndexMap
=
new
Map
<
number
,
number
>
();
this
.
useActiveGpu
=
useActiveGpu
;
}
}
}
export
function
parseGpuIndices
(
gpuIndices
?:
string
):
Set
<
number
>
|
undefined
{
export
function
parseGpuIndices
(
gpuIndices
?:
string
):
Set
<
number
>
|
undefined
{
...
...
src/nni_manager/training_service/remote_machine/remoteMachineTrainingService.ts
View file @
139e0a90
...
@@ -466,6 +466,7 @@ class RemoteMachineTrainingService implements TrainingService {
...
@@ -466,6 +466,7 @@ class RemoteMachineTrainingService implements TrainingService {
let
connectedRMNum
:
number
=
0
;
let
connectedRMNum
:
number
=
0
;
rmMetaList
.
forEach
(
async
(
rmMeta
:
RemoteMachineMeta
)
=>
{
rmMetaList
.
forEach
(
async
(
rmMeta
:
RemoteMachineMeta
)
=>
{
rmMeta
.
occupiedGpuIndexMap
=
new
Map
<
number
,
number
>
();
let
sshClientManager
:
SSHClientManager
=
new
SSHClientManager
([],
this
.
MAX_TRIAL_NUMBER_PER_SSHCONNECTION
,
rmMeta
);
let
sshClientManager
:
SSHClientManager
=
new
SSHClientManager
([],
this
.
MAX_TRIAL_NUMBER_PER_SSHCONNECTION
,
rmMeta
);
let
sshClient
:
Client
=
await
sshClientManager
.
getAvailableSSHClient
();
let
sshClient
:
Client
=
await
sshClientManager
.
getAvailableSSHClient
();
this
.
machineSSHClientMap
.
set
(
rmMeta
,
sshClientManager
);
this
.
machineSSHClientMap
.
set
(
rmMeta
,
sshClientManager
);
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment