Skip to content

GitLab

Commit 252f36f8 authored by Deshui Yu's avatar Deshui Yu
Browse files

NNI dogfood version 1

parent 781cea26
Hide whitespace changes
Inline Side-by-side
Showing with 4305 additions and 0 deletions
src/nni_manager/training_service/common/gpuData.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
/**
* GPU Infromation class
* Representing the dynamic and static information retrieved from Nvidia-smi
*/
export class GPUInfo {
// The number of active process running on this GPU
public activeProcessNum: number;
// Memory utilization of this GPU
public gpuMemUtil: number;
// GPU utilization of this GPU
public gpuUtil: number;
// the index number of this GPU (starting from 0)
public readonly index: number;
constructor(activeProcessNum : number, gpuMemUtil : number, gpuUtil : number, index : number) {
this.activeProcessNum = activeProcessNum;
this.gpuMemUtil = gpuMemUtil;
this.gpuUtil = gpuUtil;
this.index = index;
}
}
/**
* GPU Sumamry for each machine
*/
export class GPUSummary {
// GPU count on the machine
public readonly gpuCount: number;
// The timestamp when GPU summary data queried
public readonly timestamp: string;
// The array of GPU information for each GPU card
public readonly gpuInfos: GPUInfo[];
constructor(gpuCount: number, timestamp: string, gpuInfos: GPUInfo[]) {
this.gpuCount = gpuCount;
this.timestamp = timestamp;
this.gpuInfos = gpuInfos;
}
}
src/nni_manager/training_service/local/gpuScheduler.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import * as nodeNvidiaSmi from 'node-nvidia-smi';
import { delay } from '../../common/utils';
import { GPUInfo, GPUSummary } from '../common/gpuData';
/* Example of nvidia-smi result
{
"nvidia_smi_log": {
"timestamp": "Fri Jul 13 15:17:27 2018",
"driver_version": "396.26",
"attached_gpus": "8",
"gpu": [
...,
{
...
"minor_number": "5",
"utilization": {
"gpu_util": "100 %",
"memory_util": "27 %",
"encoder_util": "0 %",
"decoder_util": "0 %"
},
...
"processes": {
"process_info": {
"pid": "39943",
"type": "C",
"process_name": "python",
"used_memory": "16229 MiB"
}
},
...
},
{
"$": {
"id": "00000000:8E:00.0"
},
"product_name": "Tesla P100-PCIE-16GB",
"product_brand": "Tesla",
"display_mode": "Enabled",
"display_active": "Disabled",
"persistence_mode": "Disabled",
"accounting_mode": "Disabled",
"accounting_mode_buffer_size": "4000",
"driver_model": {
"current_dm": "N/A",
"pending_dm": "N/A"
},
"serial": "0321017108732",
"uuid": "GPU-df3e8a0a-ce99-350c-b196-c3775eb32309",
"minor_number": "6",
"vbios_version": "86.00.40.00.01",
"multigpu_board": "No",
"board_id": "0x8e00",
"gpu_part_number": "900-2H400-0300-031",
"inforom_version": {
"img_version": "H400.0201.00.08",
"oem_object": "1.1",
"ecc_object": "4.1",
"pwr_object": "N/A"
},
"gpu_operation_mode": {
"current_gom": "N/A",
"pending_gom": "N/A"
},
"gpu_virtualization_mode": {
"virtualization_mode": "None"
},
"ibmnpu": {
"relaxed_ordering_mode": "N/A"
},
"pci": {
"pci_bus": "8E",
"pci_device": "00",
"pci_domain": "0000",
"pci_device_id": "15F810DE",
"pci_bus_id": "00000000:8E:00.0",
"pci_sub_system_id": "118F10DE",
"pci_gpu_link_info": {
"pcie_gen": {
"max_link_gen": "3",
"current_link_gen": "3"
},
"link_widths": {
"max_link_width": "16x",
"current_link_width": "16x"
}
},
"pci_bridge_chip": {
"bridge_chip_type": "N/A",
"bridge_chip_fw": "N/A"
},
"replay_counter": "0",
"tx_util": "0 KB/s",
"rx_util": "0 KB/s"
},
"fan_speed": "N/A",
"performance_state": "P0",
"clocks_throttle_reasons": {
"clocks_throttle_reason_gpu_idle": "Not Active",
"clocks_throttle_reason_applications_clocks_setting": "Not Active",
"clocks_throttle_reason_sw_power_cap": "Not Active",
"clocks_throttle_reason_hw_slowdown": "Not Active",
"clocks_throttle_reason_hw_thermal_slowdown": "Not Active",
"clocks_throttle_reason_hw_power_brake_slowdown": "Not Active",
"clocks_throttle_reason_sync_boost": "Not Active",
"clocks_throttle_reason_sw_thermal_slowdown": "Not Active"
},
"fb_memory_usage": {
"total": "16280 MiB",
"used": "16239 MiB",
"free": "41 MiB"
},
"bar1_memory_usage": {
"total": "16384 MiB",
"used": "2 MiB",
"free": "16382 MiB"
},
"compute_mode": "Default",
"utilization": {
"gpu_util": "0 %",
"memory_util": "0 %",
"encoder_util": "0 %",
"decoder_util": "0 %"
},
"encoder_stats": {
"session_count": "0",
"average_fps": "0",
"average_latency": "0"
},
"ecc_mode": {
"current_ecc": "Enabled",
"pending_ecc": "Enabled"
},
"ecc_errors": {
"volatile": {
"single_bit": {
"device_memory": "0",
"register_file": "0",
"l1_cache": "N/A",
"l2_cache": "0",
"texture_memory": "0",
"texture_shm": "0",
"cbu": "N/A",
"total": "0"
},
"double_bit": {
"device_memory": "0",
"register_file": "0",
"l1_cache": "N/A",
"l2_cache": "0",
"texture_memory": "0",
"texture_shm": "0",
"cbu": "N/A",
"total": "0"
}
},
"aggregate": {
"single_bit": {
"device_memory": "0",
"register_file": "0",
"l1_cache": "N/A",
"l2_cache": "0",
"texture_memory": "0",
"texture_shm": "0",
"cbu": "N/A",
"total": "0"
},
"double_bit": {
"device_memory": "0",
"register_file": "0",
"l1_cache": "N/A",
"l2_cache": "0",
"texture_memory": "0",
"texture_shm": "0",
"cbu": "N/A",
"total": "0"
}
}
},
"retired_pages": {
"multiple_single_bit_retirement": {
"retired_count": "0",
"retired_page_addresses": "\n\t\t\t\t"
},
"double_bit_retirement": {
"retired_count": "0",
"retired_page_addresses": "\n\t\t\t\t"
},
"pending_retirement": "No"
},
"temperature": {
"gpu_temp": "33 C",
"gpu_temp_max_threshold": "85 C",
"gpu_temp_slow_threshold": "82 C",
"gpu_temp_max_gpu_threshold": "N/A",
"memory_temp": "N/A",
"gpu_temp_max_mem_threshold": "N/A"
},
"power_readings": {
"power_state": "P0",
"power_management": "Supported",
"power_draw": "37.29 W",
"power_limit": "250.00 W",
"default_power_limit": "250.00 W",
"enforced_power_limit": "250.00 W",
"min_power_limit": "125.00 W",
"max_power_limit": "250.00 W"
},
"clocks": {
"graphics_clock": "1328 MHz",
"sm_clock": "1328 MHz",
"mem_clock": "715 MHz",
"video_clock": "1189 MHz"
},
"applications_clocks": {
"graphics_clock": "1189 MHz",
"mem_clock": "715 MHz"
},
"default_applications_clocks": {
"graphics_clock": "1189 MHz",
"mem_clock": "715 MHz"
},
"max_clocks": {
"graphics_clock": "1328 MHz",
"sm_clock": "1328 MHz",
"mem_clock": "715 MHz",
"video_clock": "1328 MHz"
},
"max_customer_boost_clocks": {
"graphics_clock": "1328 MHz"
},
"clock_policy": {
"auto_boost": "N/A",
"auto_boost_default": "N/A"
},
"supported_clocks": {
"supported_mem_clock": {
"value": "715 MHz",
"supported_graphics_clock": [
"1328 MHz",
"1316 MHz",
"1303 MHz",
...
]
}
},
"processes": {
"process_info": {
"pid": "40788",
"type": "C",
"process_name": "python",
"used_memory": "16229 MiB"
}
},
"accounted_processes": "\n\t\t"
},
...
]
}
}*/
/**
* GPUScheduler
*/
class GPUScheduler {
private gpuSummary!: GPUSummary;
private stopping: boolean;
constructor() {
this.stopping = false;
}
public async run(): Promise<void> {
while (!this.stopping) {
try {
this.gpuSummary = await this.readGPUSummary();
} catch (error) {
console.error('Read GPU summary failed with error', error);
}
await delay(5000);
}
}
public getAvailableGPUIndices(): number[] {
if (this.gpuSummary !== undefined) {
return this.gpuSummary.gpuInfos.filter((info: GPUInfo) => info.activeProcessNum === 0).map((info: GPUInfo) => info.index);
}
return [];
}
public stop(): void {
this.stopping = true;
}
private readGPUSummary(): Promise<GPUSummary> {
return new Promise((resolve: Function, reject: Function): void => {
nodeNvidiaSmi((error: Error, data: nodeNvidiaSmi.GPUInfo) => {
if (error !== undefined) {
reject(error);
} else {
const gpuSummary: GPUSummary = new GPUSummary(
parseInt(data.nvidia_smi_log.attached_gpus, 10),
Date().toString(),
data.nvidia_smi_log.gpu.map((gpuInfo: {
minor_number: string;
utilization: {
gpu_util: string;
memory_util: string;
};
process: string | object;
}) => new GPUInfo(
typeof gpuInfo.process === 'object' ? 1 : 0,
parseFloat(gpuInfo.utilization.memory_util),
parseFloat(gpuInfo.utilization.gpu_util),
parseInt(gpuInfo.minor_number, 10)
))
);
resolve(gpuSummary);
}
});
});
}
}
export { GPUScheduler };
src/nni_manager/training_service/local/localTrainingService.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import * as cpp from 'child-process-promise';
import * as cp from 'child_process';
import { EventEmitter } from 'events';
import * as fs from 'fs';
import * as path from 'path';
import * as ts from 'tail-stream';
import { NNIError, NNIErrorNames } from '../../common/errors';
import { getLogger, Logger } from '../../common/log';
import {
HostJobApplicationForm, JobApplicationForm, TrainingService, TrialJobApplicationForm,
TrialJobDetail, TrialJobMetric, TrialJobStatus
} from '../../common/trainingService';
import { delay, getExperimentRootDir, uniqueString } from '../../common/utils';
const tkill = require('tree-kill');
/**
* Decode a command
* @param Buffer binary incoming data
* @returns a tuple of (success, commandType, content, remain)
* success: true if the buffer contains at least one complete command; otherwise false
* remain: remaining data after the first command
*/
function decodeCommand(data: Buffer): [boolean, string, string, Buffer] {
if (data.length < 8) {
return [false, '', '', data];
}
const commandType: string = data.slice(0, 2).toString();
const contentLength: number = parseInt(data.slice(2, 8).toString(), 10);
if (data.length < contentLength + 8) {
return [false, '', '', data];
}
const content: string = data.slice(8, contentLength + 8).toString();
const remain: Buffer = data.slice(contentLength + 8);
return [true, commandType, content, remain];
}
/**
* LocalTrialJobDetail
*/
class LocalTrialJobDetail implements TrialJobDetail {
public id: string;
public status: TrialJobStatus;
public submitTime: Date;
public startTime?: Date;
public endTime?: Date;
public tags?: string[];
public url?: string;
public workingDirectory: string;
public form: JobApplicationForm;
public pid?: number;
constructor(id: string, status: TrialJobStatus, submitTime: Date, workingDirectory: string, form: JobApplicationForm) {
this.id = id;
this.status = status;
this.submitTime = submitTime;
this.workingDirectory = workingDirectory;
this.form = form;
this.url = `file://localhost:${workingDirectory}`;
}
}
/**
* Local training service
*/
class LocalTrainingService implements TrainingService {
private eventEmitter: EventEmitter;
private jobMap: Map<string, LocalTrialJobDetail>;
private jobQueue: string[];
private initialized: boolean;
private stopping: boolean;
private rootDir!: string;
private codeDir!: string;
private command!: string;
private log: Logger;
constructor() {
this.eventEmitter = new EventEmitter();
this.jobMap = new Map<string, LocalTrialJobDetail>();
this.jobQueue = [];
this.initialized = false;
this.stopping = false;
this.log = getLogger();
}
public async run(): Promise<void> {
while (!this.stopping) {
while (this.jobQueue.length !== 0) {
const trialJobId: string = this.jobQueue[0];
const [success, resource] = this.tryGetAvailableResource();
if (!success) {
break;
}
this.occupyResource(resource);
this.jobQueue.shift();
await this.runTrialJob(trialJobId, resource);
}
await delay(5000);
}
}
public async listTrialJobs(): Promise<TrialJobDetail[]> {
const jobs: TrialJobDetail[] = [];
for (const key of this.jobMap.keys()) {
const trialJob: TrialJobDetail = await this.getTrialJob(key);
if (trialJob.form.jobType === 'TRIAL') {
jobs.push(trialJob);
}
}
return jobs;
}
public async getTrialJob(trialJobId: string): Promise<TrialJobDetail> {
const trialJob: LocalTrialJobDetail | undefined = this.jobMap.get(trialJobId);
if (trialJob === undefined) {
throw new NNIError(NNIErrorNames.NOT_FOUND, 'Trial job not found');
}
if (trialJob.form.jobType === 'HOST') {
return this.getHostJob(trialJobId);
}
if (trialJob.status === 'RUNNING') {
let alive: boolean = false;
try {
await cpp.exec(`kill -0 ${trialJob.pid}`);
alive = true;
} catch (error) {
//ignore
}
if (!alive) {
trialJob.endTime = new Date();
this.setTrialJobStatus(trialJob, 'FAILED');
try {
const state: string = await fs.promises.readFile(path.join(trialJob.workingDirectory, '.nni', 'state'), 'utf8');
const match: RegExpMatchArray | null = state.trim().match(/^(\d+)\s+(\d+)$/);
if (match !== null) {
const { 1: code, 2: timestamp } = match;
if (parseInt(code, 10) === 0) {
this.setTrialJobStatus(trialJob, 'SUCCEEDED');
}
trialJob.endTime = new Date(parseInt(timestamp, 10));
}
} catch (error) {
//ignore
}
this.log.info(`trailJob status update: ${trialJobId}, ${trialJob.status}`);
}
}
return trialJob;
}
public addTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void {
this.eventEmitter.on('metric', listener);
}
public removeTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void {
this.eventEmitter.off('metric', listener);
}
public submitTrialJob(form: JobApplicationForm): Promise<TrialJobDetail> {
this.log.info(`submitTrialJob: form: ${JSON.stringify(form)}`);
if (form.jobType === 'HOST') {
return this.runHostJob(<HostJobApplicationForm>form);
} else if (form.jobType === 'TRIAL') {
const trialJobId: string = uniqueString(5);
const trialJobDetail: LocalTrialJobDetail = new LocalTrialJobDetail(
trialJobId,
'WAITING',
new Date(),
path.join(this.rootDir, 'trials', trialJobId),
form);
this.jobQueue.push(trialJobId);
this.jobMap.set(trialJobId, trialJobDetail);
this.log.debug(`submitTrialJob: return: ${JSON.stringify(trialJobDetail)} `);
return Promise.resolve(trialJobDetail);
} else {
return Promise.reject(new Error(`Job form not supported: ${JSON.stringify(form)}`));
}
}
public async cancelTrialJob(trialJobId: string): Promise<void> {
this.log.info(`cancelTrialJob: ${trialJobId}`);
const trialJob: LocalTrialJobDetail | undefined = this.jobMap.get(trialJobId);
if (trialJob === undefined) {
throw new NNIError(NNIErrorNames.NOT_FOUND, 'Trial job not found');
}
if (trialJob.form.jobType === 'TRIAL') {
await tkill(trialJob.pid, 'SIGKILL');
} else if (trialJob.form.jobType === 'HOST') {
await cpp.exec(`pkill -9 -P ${trialJob.pid}`);
} else {
throw new Error(`Job type not supported: ${trialJob.form.jobType}`);
}
this.setTrialJobStatus(trialJob, 'USER_CANCELED');
}
public async setClusterMetadata(key: string, value: string): Promise<void> {
if (!this.initialized) {
this.rootDir = getExperimentRootDir();
await cpp.exec(`mkdir -p ${this.rootDir}`);
this.initialized = true;
}
switch (key) {
case 'codeDir':
this.codeDir = value;
break;
case 'command':
this.command = value;
break;
default:
}
}
public getClusterMetadata(key: string): Promise<string> {
switch (key) {
case 'codeDir':
return Promise.resolve(this.codeDir);
case 'command':
return Promise.resolve(this.command);
default:
return Promise.reject(new NNIError(NNIErrorNames.NOT_FOUND, 'Key not found'));
}
}
public cleanUp(): Promise<void> {
this.stopping = true;
return Promise.resolve();
}
protected onTrialJobStatusChanged(trialJob: TrialJobDetail, oldStatus: TrialJobStatus): void {
//abstract
}
protected getEnvironmentVariables(trialJobDetail: TrialJobDetail, _: {}): { key: string; value: string }[] {
return [
{ key: 'NNI_PLATFORM', value: 'local' },
{ key: 'NNI_SYS_DIR', value: trialJobDetail.workingDirectory },
{ key: 'NNI_TRIAL_JOB_ID', value: trialJobDetail.id },
{ key: 'NNI_OUTPUT_DIR', value: trialJobDetail.workingDirectory }
];
}
protected setExtraProperties(trialJobDetail: TrialJobDetail, resource: {}): void {
//abstract
}
protected tryGetAvailableResource(): [boolean, {}] {
return [true, {}];
}
protected occupyResource(_: {}): void {
//abstract
}
private setTrialJobStatus(trialJob: LocalTrialJobDetail, newStatus: TrialJobStatus): void {
if (trialJob.status !== newStatus) {
const oldStatus: TrialJobStatus = trialJob.status;
trialJob.status = newStatus;
this.onTrialJobStatusChanged(trialJob, oldStatus);
}
}
private async runTrialJob(trialJobId: string, resource: {}): Promise<void> {
const trialJobDetail: LocalTrialJobDetail = <LocalTrialJobDetail>this.jobMap.get(trialJobId);
const variables: { key: string; value: string }[] = this.getEnvironmentVariables(trialJobDetail, resource);
const runScriptLines: string[] = [];
runScriptLines.push(
'#!/bin/bash',
`cd ${this.codeDir}`);
for (const variable of variables) {
runScriptLines.push(`export ${variable.key}=${variable.value}`);
}
runScriptLines.push(
`eval ${this.command} 2>${path.join(trialJobDetail.workingDirectory, '.nni', 'stderr')}`,
`echo $? \`date +%s%3N\` >${path.join(trialJobDetail.workingDirectory, '.nni', 'state')}`);
await cpp.exec(`mkdir -p ${trialJobDetail.workingDirectory}`);
await cpp.exec(`mkdir -p ${path.join(trialJobDetail.workingDirectory, '.nni')}`);
await cpp.exec(`touch ${path.join(trialJobDetail.workingDirectory, '.nni', 'metrics')}`);
await fs.promises.writeFile(path.join(trialJobDetail.workingDirectory, 'run.sh'), runScriptLines.join('\n'), { encoding: 'utf8' });
await fs.promises.writeFile(
path.join(trialJobDetail.workingDirectory, 'parameter.cfg'),
(<TrialJobApplicationForm>trialJobDetail.form).hyperParameters,
{ encoding: 'utf8' });
const process: cp.ChildProcess = cp.exec(`bash ${path.join(trialJobDetail.workingDirectory, 'run.sh')}`);
this.setTrialJobStatus(trialJobDetail, 'RUNNING');
trialJobDetail.startTime = new Date();
trialJobDetail.pid = process.pid;
this.setExtraProperties(trialJobDetail, resource);
let buffer: Buffer = Buffer.alloc(0);
const stream: ts.Stream = ts.createReadStream(path.join(trialJobDetail.workingDirectory, '.nni', 'metrics'));
stream.on('data', (data: Buffer) => {
buffer = Buffer.concat([buffer, data]);
while (buffer.length > 0) {
const [success, , content, remain] = decodeCommand(buffer);
if (!success) {
break;
}
this.eventEmitter.emit('metric', {
id: trialJobDetail.id,
data: content
});
this.log.debug(`Sending metrics, job id: ${trialJobDetail.id}, metrics: ${content}`);
buffer = remain;
}
});
}
private async runHostJob(form: HostJobApplicationForm): Promise<TrialJobDetail> {
const jobId: string = uniqueString(5);
const workDir: string = path.join(this.rootDir, 'hostjobs', jobId);
await cpp.exec(`mkdir -p ${workDir}`);
const wrappedCmd: string = `cd ${workDir} && ${form.cmd}>stdout 2>stderr`;
this.log.debug(`runHostJob: command: ${wrappedCmd}`);
const process: cp.ChildProcess = cp.exec(wrappedCmd);
const jobDetail: LocalTrialJobDetail = {
id: jobId,
status: 'RUNNING',
submitTime: new Date(),
workingDirectory: workDir,
form: form,
pid: process.pid
};
this.jobMap.set(jobId, jobDetail);
this.log.debug(`runHostJob: return: ${JSON.stringify(jobDetail)} `);
return jobDetail;
}
private async getHostJob(jobId: string): Promise<TrialJobDetail> {
const jobDetail: LocalTrialJobDetail | undefined = this.jobMap.get(jobId);
if (jobDetail === undefined) {
throw new NNIError(NNIErrorNames.NOT_FOUND, `Host Job not found: ${jobId}`);
}
try {
await cpp.exec(`kill -0 ${jobDetail.pid}`);
return jobDetail;
} catch (error) {
if (error instanceof Error) {
this.log.debug(`getHostJob: error: ${error.message}`);
this.jobMap.delete(jobId);
throw new NNIError(NNIErrorNames.NOT_FOUND, `Host Job not found: ${error.message}`);
} else {
throw error;
}
}
}
}
export { LocalTrainingService };
src/nni_manager/training_service/local/localTrainingServiceForGPU.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import { TrialJobDetail, TrialJobStatus } from '../../common/trainingService';
import { GPUScheduler } from './gpuScheduler';
import { LocalTrainingService } from './localTrainingService';
type LocalTrialJobDetailForGPU = TrialJobDetail & { gpuIndices: number[] };
/**
* Local training service for GPU
*/
class LocalTrainingServiceForGPU extends LocalTrainingService {
private requiredGPUNum!: number;
private gpuScheduler!: GPUScheduler;
private availableGPUIndices: boolean[];
constructor() {
super();
this.availableGPUIndices = Array(16).fill(false); // Assume the maximum gpu number is 16
}
public async run(): Promise<void> {
if (this.gpuScheduler !== undefined) {
await Promise.all([
this.gpuScheduler.run(),
super.run()
]);
} else {
await super.run();
}
}
public async setClusterMetadata(key: string, value: string): Promise<void> {
await super.setClusterMetadata(key, value);
switch (key) {
case 'requiredGPUNum':
this.requiredGPUNum = parseInt(value, 10);
if (this.gpuScheduler === undefined) {
this.gpuScheduler = new GPUScheduler();
}
break;
default:
}
}
public getClusterMetadata(key: string): Promise<string> {
switch (key) {
case 'requiredGPUNum':
return Promise.resolve(`${this.requiredGPUNum}`);
default:
return super.getClusterMetadata(key);
}
}
public cleanUp(): Promise<void> {
if (this.gpuScheduler !== undefined) {
this.gpuScheduler.stop();
}
return super.cleanUp();
}
protected onTrialJobStatusChanged(trialJob: LocalTrialJobDetailForGPU, oldStatus: TrialJobStatus): void {
if (trialJob.gpuIndices.length !== 0) {
if (oldStatus === 'RUNNING' && trialJob.status !== 'RUNNING') {
for (const index of trialJob.gpuIndices) {
this.availableGPUIndices[index] = false;
}
}
}
}
protected getEnvironmentVariables(
trialJobDetail: TrialJobDetail,
resource: { gpuIndices: number[] }): { key: string; value: string }[] {
const variables: { key: string; value: string }[] = super.getEnvironmentVariables(trialJobDetail, resource);
variables.push({
key: 'CUDA_VISIBLE_DEVICES',
value: resource.gpuIndices.join(',')
});
return variables;
}
protected setExtraProperties(trialJobDetail: LocalTrialJobDetailForGPU, resource: { gpuIndices: number[] }): void {
super.setExtraProperties(trialJobDetail, resource);
trialJobDetail.gpuIndices = resource.gpuIndices;
}
protected tryGetAvailableResource(): [boolean, {}] {
const [success, resource] = super.tryGetAvailableResource();
if (!success || this.gpuScheduler === undefined) {
return [success, resource];
}
const availableGPUIndices: number[] = this.gpuScheduler.getAvailableGPUIndices();
const selectedGPUIndices: number[] = availableGPUIndices.filter((index: number) => this.availableGPUIndices[index] === false);
if (selectedGPUIndices.length < this.requiredGPUNum) {
return [false, resource];
}
selectedGPUIndices.splice(this.requiredGPUNum);
Object.assign(resource, { gpuIndices: selectedGPUIndices });
return [true, resource];
}
protected occupyResource(resource: { gpuIndices: number[] }): void {
super.occupyResource(resource);
for (const index of resource.gpuIndices) {
this.availableGPUIndices[index] = true;
}
}
}
export { LocalTrainingServiceForGPU };
src/nni_manager/training_service/remote_machine/gpuScheduler.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import { Client } from 'ssh2';
import { Deferred } from 'ts-deferred';
import { getLogger, Logger } from '../../common/log';
import { GPUInfo } from '../common/gpuData';
import { RemoteMachineMeta, RemoteMachineScheduleResult, RemoteMachineScheduleInfo, ScheduleResultType } from './remoteMachineData';
/**
* A simple GPU scheduler implementation
*/
export class GPUScheduler {
private readonly machineSSHClientMap : Map<RemoteMachineMeta, Client>;
private log: Logger = getLogger();
/**
* Constructor
* @param machineSSHClientMap map from remote machine to sshClient
*/
constructor(machineSSHClientMap : Map<RemoteMachineMeta, Client>) {
this.machineSSHClientMap = machineSSHClientMap;
}
/**
* Schedule a machine according to the constraints (requiredGPUNum)
* @param requiredGPUNum required GPU number
*/
public scheduleMachine(requiredGPUNum : Number | undefined, trialJobId : string) : RemoteMachineScheduleResult {
const deferred: Deferred<RemoteMachineScheduleResult> = new Deferred<RemoteMachineScheduleResult>();
let scheduleResult : RemoteMachineScheduleResult = {
resultType : ScheduleResultType.TMP_NO_AVAILABLE_GPU,
scheduleInfo : undefined
};
// Step 0: Check if required GPU number not exceeds the total GPU number in all machines
const eligibleRM : RemoteMachineMeta[] = Array.from(this.machineSSHClientMap.keys()).filter((rmMeta : RemoteMachineMeta) =>
rmMeta.gpuSummary === undefined || requiredGPUNum === undefined || rmMeta.gpuSummary.gpuCount >= requiredGPUNum );
if(eligibleRM.length == 0) {
// If the required gpu number exceeds the upper limit of all machine's GPU number
// Return REQUIRE_EXCEED_TOTAL directly
return ({
resultType : ScheduleResultType.REQUIRE_EXCEED_TOTAL,
scheduleInfo : undefined
});
}
// Step 1: Generate GPU resource map for remote machines
const totalResourceMap : Map<RemoteMachineMeta, GPUInfo[]> = this.gpuResourceDetection(requiredGPUNum);
// Step 2: Find machine whose GPU can be allocated based on user GPU requirement, and allocate GPU
for (const rmMeta of Array.from(totalResourceMap.keys())) {
const gpuInfos : GPUInfo[] | undefined = totalResourceMap.get(rmMeta);
if(gpuInfos !== undefined && (requiredGPUNum === undefined || gpuInfos.length >= requiredGPUNum)) {
const allocatedGPUIndex : number[] = Array();
// Allocate
gpuInfos.forEach((gpuInfo : GPUInfo) => {
rmMeta.gpuReservation.set(gpuInfo.index, trialJobId);
allocatedGPUIndex.push(gpuInfo.index);
});
// Construct scheduling return object
const sshClient : Client | undefined = this.machineSSHClientMap.get(rmMeta);
if(sshClient !== undefined) {
this.log.info(`Found available machine, trialJobId is ${trialJobId}, ip is ${rmMeta.ip}, gpu allocated is ${allocatedGPUIndex.toString()}`);
// We found the first available machine whose GPU resource can match user requirement
return {
resultType : ScheduleResultType.SUCCEED,
scheduleInfo : {
rmMeta : rmMeta,
client : sshClient,
cuda_visible_device : allocatedGPUIndex.join(',')
}
};
}
}
}
// Step 3: If not found machine whose GPU is availabe, then find the first machine whose GPU summary is unknown
for (const rmMeta of Array.from(this.machineSSHClientMap.keys())) {
const client : Client | undefined = this.machineSSHClientMap.get(rmMeta);
if(rmMeta.gpuSummary == undefined && client !== undefined) {
// We found the firstmachine whose GPU summary is unknown
return {
resultType : ScheduleResultType.SUCCEED,
scheduleInfo :{
rmMeta : rmMeta,
client : client,
//Since gpu information is unknown, make all GPU resources visible to the job
cuda_visible_device : ''
}
};
}
};
this.log.warning(`Scheduler: trialJob id ${trialJobId}, no machine can be scheduled, resolve as TMP_NO_AVAILABLE_GPU `);
// Otherwise, no machine can be scheduled, resolve as TMP_NO_AVAILABLE_GPU
return {
resultType : ScheduleResultType.TMP_NO_AVAILABLE_GPU,
scheduleInfo : undefined
};
}
/**
* Detect available GPU resource for a remote machine
* @param rmMeta Remote machine metadata
* @param requiredGPUNum required GPU number by application
* @param availableGPUMap available GPU resource filled by this detection
* @returns Available GPU number on this remote machine
*/
private gpuResourceDetection(requiredGPUNum : Number | undefined) : Map<RemoteMachineMeta, GPUInfo[]> {
const totalResourceMap : Map<RemoteMachineMeta, GPUInfo[]> = new Map<RemoteMachineMeta, GPUInfo[]>();
this.machineSSHClientMap.forEach((client: Client, rmMeta: RemoteMachineMeta) => {
// Assgin totoal GPU count as init available GPU number
if(rmMeta.gpuSummary !== undefined) {
const availableGPUs : GPUInfo[] = Array();
if(rmMeta.gpuReservation === undefined) {
rmMeta.gpuReservation = new Map<number, string>();
}
const gpuReservation = rmMeta.gpuReservation;
rmMeta.gpuSummary.gpuInfos.forEach((gpuInfo: GPUInfo) => {
//this.log.info(`GPU index:${gpuInfo.index}, activeProcessNum is ${gpuInfo.activeProcessNum}, GPU reservation is ${JSON.stringify([...gpuReservation])}`);
// if the GPU has active process, OR be reserved by a job,
// We should NOT allocate this GPU
if (gpuInfo.activeProcessNum === 0
&& !gpuReservation.has(gpuInfo.index)
&& requiredGPUNum !== undefined
&& availableGPUs.length < requiredGPUNum) {
availableGPUs.push(gpuInfo);
}
});
totalResourceMap.set(rmMeta, availableGPUs);
}
});
return totalResourceMap;
}
}
src/nni_manager/training_service/remote_machine/metricsCollector.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import * as assert from 'assert';
import { EventEmitter } from 'events';
import * as path from 'path';
import { Client } from 'ssh2';
import { getLogger, Logger } from '../../common/log';
import { TrialJobStatus } from '../../common/trainingService';
import { JobMetrics, RemoteCommandResult, RemoteMachineMeta, RemoteMachineTrialJobDetail } from './remoteMachineData';
import { SSHClientUtility } from './sshClientUtility';
export class MetricsCollector {
private machineSSHClientMap : Map<RemoteMachineMeta, Client>;
private trialJobsMap : Map<string, any>;
private expRootDir: string;
private metricsEmitter: EventEmitter;
private log: Logger = getLogger();
constructor(clientMap: Map<RemoteMachineMeta, Client>,
jobMap: Map<string, any>,
expDir: string, eventEmitter: EventEmitter) {
this.machineSSHClientMap = clientMap;
this.trialJobsMap = jobMap;
this.expRootDir = expDir;
this.metricsEmitter = eventEmitter;
}
public async collectMetrics(): Promise<void> {
const aliveJobStatus : TrialJobStatus[] = ['RUNNING', 'SUCCEEDED'];
const runningJobsMap: Map<RemoteMachineMeta, string[]> = this.getTrialJobIdsGroupByRmMeta(aliveJobStatus);
const readMetricsTasks: Promise<JobMetrics[]>[] = [];;
runningJobsMap.forEach((jobIds: string[], rmMeta: RemoteMachineMeta) => {
readMetricsTasks.push(this.readRmMetrics(rmMeta, jobIds));
});
const allMetrics = await Promise.all(readMetricsTasks.map(task => { return task.catch(err => { this.log.error(err.message); }); }));
allMetrics.forEach((rmMetrics) => {
if (rmMetrics !== undefined && rmMetrics.length > 0) {
rmMetrics.forEach((jobMetrics) => {
const trialJobId : string = jobMetrics.jobId;
// If job status is not alive again, remove its GPU reservation
if(!['RUNNING'].includes(jobMetrics.jobStatus)) {
runningJobsMap.forEach((jobIds: string[], rmMeta: RemoteMachineMeta) => {
// If remote machine has no GPU, gpuReservcation is not initialized, so check if it's undefined
if(rmMeta.gpuReservation !== undefined) {
rmMeta.gpuReservation.forEach((reserveTrialJobId : string, gpuIndex : number) => {
if(reserveTrialJobId == trialJobId) {
rmMeta.gpuReservation.delete(gpuIndex);
}
});
}
});
}
this.sendMetricsToListeners(jobMetrics);
});
}
});
}
private getTrialJobIdsGroupByRmMeta(status: TrialJobStatus[]): Map<RemoteMachineMeta, string[]> {
const map: Map<RemoteMachineMeta, string[]> = new Map<RemoteMachineMeta, string[]>();
this.trialJobsMap.forEach((trialJob, id) => {
if (status.includes(trialJob.status)) {
if (map.has(trialJob.rmMeta)) {
const ids = map.get(trialJob.rmMeta);
if (ids !== undefined) {
ids.push(id);
}
} else {
map.set(trialJob.rmMeta, [id]);
}
}
});
return map;
}
private sendMetricsToListeners(jobMetrics: JobMetrics): void {
if (jobMetrics === undefined) {
return;
}
const jobId: string = jobMetrics.jobId;
jobMetrics.metrics.forEach((metric: string) => {
if (metric.length > 0) {
this.metricsEmitter.emit('metric', {
id : jobId,
data : metric
});
}
});
}
private async readRmMetrics(rmMeta: RemoteMachineMeta, trialJobIds: string[]): Promise<JobMetrics[]> {
if (trialJobIds === undefined || trialJobIds.length < 1) {
return [];
}
const scriptFile: string = path.join(path.dirname(path.dirname(this.expRootDir)), 'scripts', 'metrics_reader.py');
const cmdStr: string = `python3 ${scriptFile} --experiment_dir ${this.expRootDir} --trial_job_ids ${trialJobIds.join(',')}`;
trialJobIds.forEach((id: string) => {
const trialJob: RemoteMachineTrialJobDetail = this.trialJobsMap.get(id);
assert(trialJob.rmMeta === rmMeta);
});
const sshClient: Client | undefined = this.machineSSHClientMap.get(rmMeta);
if (sshClient === undefined) {
throw new Error('SSHClient not found!');
}
const result: RemoteCommandResult = await SSHClientUtility.remoteExeCommand(cmdStr, sshClient);
if (result.exitCode !== 0) {
throw new Error(`Failed to read metrics data: ${result.stderr}`);
} else {
if (result.stdout !== undefined && result.stdout.length > 0) {
return <JobMetrics[]>JSON.parse(result.stdout);
} else {
return [];
}
}
}
}
src/nni_manager/training_service/remote_machine/remoteMachineData.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import { Client } from 'ssh2';
import { JobApplicationForm, TrialJobDetail, TrialJobStatus } from '../../common/trainingService';
import { GPUSummary } from '../common/gpuData';
/**
* Enum of key for remote machine metadata for configuration
*/
export enum RemoteMachineMetadataKey {
MACHINE_LIST = 'machine_list',
TRIAL_CONFIG = 'trial_config',
EXPERIMENT_ID = 'experimentId',
RANDOM_SCHEDULER = 'random_scheduler'
}
/**
* Metadata of remote machine for configuration and statuc query
*/
export class RemoteMachineMeta {
public readonly ip : string;
public readonly port : number;
public readonly username : string;
public readonly passwd: string;
public gpuSummary : GPUSummary | undefined;
/* GPU Reservation info, the key is GPU index, the value is the job id which reserves this GPU*/
public gpuReservation : Map<number, string>;
constructor(ip : string, port : number, username : string, passwd : string) {
this.ip = ip;
this.port = port;
this.username = username;
this.passwd = passwd;
this.gpuReservation = new Map<number, string>();
}
}
/**
* Configuration for trial job on remote machine
*/
export class RemoteMachineTrialConfig {
public readonly command : string;
public readonly codeDir : string;
public readonly gpuNum : number;
constructor(command : string, codeDir : string, gpuNum : number) {
this.command = command;
this.codeDir = codeDir;
this.gpuNum = gpuNum;
}
}
/**
* The execution result for command executed on remote machine
*/
export class RemoteCommandResult {
public readonly stdout : string;
public readonly stderr : string;
public readonly exitCode : number;
constructor(stdout : string, stderr : string, exitCode : number) {
this.stdout = stdout;
this.stderr = stderr;
this.exitCode = exitCode;
}
}
// tslint:disable-next-line:max-classes-per-file
export class JobMetrics {
public readonly jobId: string;
public readonly metrics: string[];
public readonly jobStatus: TrialJobStatus;
public readonly endTimestamp: number;
constructor(jobId : string, metrics : string[], jobStatus : TrialJobStatus, endTimestamp : number) {
this.jobId = jobId;
this.metrics = metrics;
this.jobStatus = jobStatus;
this.endTimestamp = endTimestamp;
}
}
/**
* RemoteMachineTrialJobDetail
*/
// tslint:disable-next-line:max-classes-per-file
export class RemoteMachineTrialJobDetail implements TrialJobDetail {
public id: string;
public status: TrialJobStatus;
public submitTime: Date;
public startTime?: Date;
public endTime?: Date;
public tags?: string[];
public url?: string;
public workingDirectory: string;
public form: JobApplicationForm;
public rmMeta?: RemoteMachineMeta;
constructor(id: string, status: TrialJobStatus, submitTime: Date, workingDirectory: string, form: JobApplicationForm) {
this.id = id;
this.status = status;
this.submitTime = submitTime;
this.workingDirectory = workingDirectory;
this.form = form;
this.tags = [];
}
}
export type RemoteMachineScheduleResult = { scheduleInfo : RemoteMachineScheduleInfo | undefined, resultType : ScheduleResultType};
export type RemoteMachineScheduleInfo = { client: Client; rmMeta : RemoteMachineMeta; cuda_visible_device : string};
export enum ScheduleResultType {
/* Schedule succeeded*/
SUCCEED,
/* Temporarily, no enough available GPU right now */
TMP_NO_AVAILABLE_GPU,
/* Cannot match requirement even if all GPU are a*/
REQUIRE_EXCEED_TOTAL
}
export const REMOTEMACHINERUNSHELLFORMAT: string =
`#!/bin/bash
export NNI_PLATFORM=remote NNI_SYS_DIR={0} NNI_TRIAL_JOB_ID={1} NNI_OUTPUT_DIR={0}
cd $NNI_SYS_DIR
echo $$ >{2}
eval {3}{4} 2>{5}
echo $? \`date +%s%3N\` >{6}`;
export const HOSTJOBSHELLFORMAT: string =
`#!/bin/bash
cd {0}
echo $$ >{1}
eval {2} >stdout 2>stderr
echo $? \`date +%s%3N\` >{3}`;
src/nni_manager/training_service/remote_machine/remoteMachineTrainingService.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import * as cpp from 'child-process-promise';
import { EventEmitter } from 'events';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import { Client } from 'ssh2';
import { Deferred } from 'ts-deferred';
import { String } from 'typescript-string-operations';
import * as component from '../../common/component';
import { NNIError, NNIErrorNames } from '../../common/errors';
import { getExperimentId } from '../../common/experimentStartupInfo';
import { getLogger, Logger } from '../../common/log';
import { ObservableTimer } from '../../common/observableTimer';
import {
HostJobApplicationForm, JobApplicationForm, TrainingService, TrialJobApplicationForm, TrialJobDetail, TrialJobMetric
} from '../../common/trainingService';
import { delay, getExperimentRootDir, uniqueString } from '../../common/utils';
import { GPUSummary } from '../common/gpuData';
import { GPUScheduler } from './gpuScheduler';
import { MetricsCollector } from './metricsCollector';
import {
HOSTJOBSHELLFORMAT, RemoteCommandResult, RemoteMachineMeta, RemoteMachineMetadataKey,
REMOTEMACHINERUNSHELLFORMAT, RemoteMachineScheduleInfo, RemoteMachineScheduleResult,
RemoteMachineTrialConfig, RemoteMachineTrialJobDetail, ScheduleResultType
} from './remoteMachineData';
import { SSHClientUtility } from './sshClientUtility';
/**
* Training Service implementation for Remote Machine (Linux)
*/
class RemoteMachineTrainingService implements TrainingService {
private machineSSHClientMap: Map<RemoteMachineMeta, Client>;
private trialJobsMap: Map<string, RemoteMachineTrialJobDetail>;
private experimentId: string | undefined;
// Experiment root directory
private expRootDir: string;
private remoteExpRootDir: string;
private trialConfig: RemoteMachineTrialConfig | undefined;
private gpuScheduler: GPUScheduler;
private jobQueue: string[];
private timer: ObservableTimer;
private stopping: boolean = false;
private metricsEmitter: EventEmitter;
private log: Logger;
constructor(@component.Inject timer: ObservableTimer) {
this.metricsEmitter = new EventEmitter();
this.trialJobsMap = new Map<string, RemoteMachineTrialJobDetail>();
this.machineSSHClientMap = new Map<RemoteMachineMeta, Client>();
this.gpuScheduler = new GPUScheduler(this.machineSSHClientMap);
this.jobQueue = [];
this.experimentId = getExperimentId();
this.expRootDir = getExperimentRootDir();
this.remoteExpRootDir = this.getRemoteModeExperimentRootDir();
this.timer = timer;
this.log = getLogger();
}
/**
* Loop to launch trial jobs and collect trial metrics
*/
public async run(): Promise<void> {
while (!this.stopping) {
while (this.jobQueue.length > 0) {
const trialJobId: string = this.jobQueue[0];
const prepareResult : boolean = await this.prepareTrialJob(trialJobId);
if (prepareResult) {
// Remove trial job with trialJobId from job queue
this.jobQueue.shift();
} else {
// Break the while loop since no GPU resource is available right now,
// Wait to schedule job in next time iteration
break;
}
};
const metricsCollector: MetricsCollector = new MetricsCollector(
this.machineSSHClientMap, this.trialJobsMap, this.remoteExpRootDir, this.metricsEmitter);
await metricsCollector.collectMetrics();
await delay(3000);
}
}
/**
* List submitted trial jobs
*/
public listTrialJobs(): Promise<TrialJobDetail[]> {
const jobs: TrialJobDetail[] = [];
const deferred: Deferred<TrialJobDetail[]> = new Deferred<TrialJobDetail[]>();
this.trialJobsMap.forEach(async (value: RemoteMachineTrialJobDetail, key: string) => {
if (value.form.jobType === 'TRIAL') {
jobs.push(await this.getTrialJob(key));
}
});
deferred.resolve(jobs);
return deferred.promise;
}
/**
* Get trial job detail information
* @param trialJobId ID of trial job
*/
public async getTrialJob(trialJobId: string): Promise<TrialJobDetail> {
const trialJob: RemoteMachineTrialJobDetail | undefined = this.trialJobsMap.get(trialJobId);
if (!trialJob) {
throw new NNIError(NNIErrorNames.NOT_FOUND, `trial job id ${trialJobId} not found`);
}
//TO DO: add another job status, and design new job status change logic
if (trialJob.status === 'RUNNING' || trialJob.status === 'UNKNOWN') {
// Get ssh client where the job is running
if (trialJob.rmMeta === undefined) {
throw new Error(`rmMeta not set for submitted job ${trialJobId}`);
}
const sshClient: Client | undefined = this.machineSSHClientMap.get(trialJob.rmMeta);
if (!sshClient) {
throw new Error(`Invalid job id: ${trialJobId}, cannot find ssh client`);
}
return this.updateTrialJobStatus(trialJob, sshClient);
} else {
return trialJob;
}
}
/**
* Add job metrics listener
* @param listener callback listener
*/
public addTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void {
this.metricsEmitter.on('metric', listener);
}
/**
* Remove job metrics listener
* @param listener callback listener
*/
public removeTrialJobMetricListener(listener: (metric: TrialJobMetric) => void): void {
this.metricsEmitter.off('metric', listener);
}
/**
* Submit trial job
* @param form trial job description form
*/
public submitTrialJob(form: JobApplicationForm): Promise<TrialJobDetail> {
this.log.info(`submitTrialJob: form: ${JSON.stringify(form)}`);
if (!this.trialConfig) {
throw new Error('trial config is not initialized');
}
if (form.jobType === 'HOST') {
return this.runHostJob(<HostJobApplicationForm>form);
} else if (form.jobType === 'TRIAL') {
// Generate trial job id(random)
const trialJobId: string = uniqueString(5);
const trialWorkingFolder: string = path.join(this.remoteExpRootDir, 'trials', trialJobId);
const trialJobDetail: RemoteMachineTrialJobDetail = new RemoteMachineTrialJobDetail(
trialJobId,
'WAITING',
new Date(),
trialWorkingFolder,
form);
this.jobQueue.push(trialJobId);
this.trialJobsMap.set(trialJobId, trialJobDetail);
return Promise.resolve(trialJobDetail);
} else {
return Promise.reject(new Error(`Job form not supported: ${JSON.stringify(form)}, jobType should be HOST or TRIAL.`));
}
}
/**
* Cancel trial job
* @param trialJobId ID of trial job
*/
public async cancelTrialJob(trialJobId: string): Promise<void> {
this.log.info(`cancelTrialJob: jobId: ${trialJobId}`);
const deferred: Deferred<void> = new Deferred<void>();
const trialJob: RemoteMachineTrialJobDetail | undefined = this.trialJobsMap.get(trialJobId);
if (!trialJob) {
deferred.reject();
throw new Error(`trial job id ${trialJobId} not found`);
}
// Remove the job with trialJobId from job queue
const index : number = this.jobQueue.indexOf(trialJobId);
if(index >= 0) {
this.jobQueue.splice(index, 1);
}
// Get ssh client where the job is running
if (trialJob.rmMeta !== undefined) {
// If the trial job is already scheduled, check its status and kill the trial process in remote machine
const sshClient: Client | undefined = this.machineSSHClientMap.get(trialJob.rmMeta);
if (!sshClient) {
deferred.reject();
throw new Error(`Invalid job id ${trialJobId}, cannot find ssh client`);
}
const jobpidPath: string = this.getJobPidPath(trialJob.id);
try {
await SSHClientUtility.remoteExeCommand(`pkill -P \`cat ${jobpidPath}\``, sshClient);
trialJob.status = 'USER_CANCELED';
} catch (error) {
// Not handle the error since pkill failed will not impact trial job's current status
this.log.error(`remoteTrainingService.cancelTrialJob: ${error.message}`);
}
} else {
// Job is not scheduled yet, set status to 'USER_CANCELLED' directly
trialJob.status = 'USER_CANCELED';
}
}
/**
* Set culster metadata
* @param key metadata key
* //1. MACHINE_LIST -- create ssh client connect of machine list
* //2. TRIAL_CONFIG -- trial configuration
* @param value metadata value
*/
public async setClusterMetadata(key: string, value: string): Promise<void> {
switch (key) {
case RemoteMachineMetadataKey.MACHINE_LIST:
await this.setupConnections(value);
break;
case RemoteMachineMetadataKey.TRIAL_CONFIG:
const remoteMachineTrailConfig: RemoteMachineTrialConfig = <RemoteMachineTrialConfig>JSON.parse(value);
// Parse trial config failed, throw Error
if (!remoteMachineTrailConfig) {
throw new Error('trial config parsed failed');
}
// codeDir is not a valid directory, throw Error
if (!fs.lstatSync(remoteMachineTrailConfig.codeDir).isDirectory()) {
throw new Error(`codeDir ${remoteMachineTrailConfig.codeDir} is not a directory`);
}
this.trialConfig = remoteMachineTrailConfig;
break;
default:
//Reject for unknown keys
throw new Error(`Uknown key: ${key}`);
}
}
/**
* Get culster metadata
* @param key metadata key
*/
public getClusterMetadata(key: string): Promise<string> {
const deferred: Deferred<string> = new Deferred<string>();
return deferred.promise;
}
public cleanUp(): Promise<void> {
this.stopping = true;
return Promise.resolve();
}
private async setupConnections(machineList: string): Promise<void> {
const deferred: Deferred<void> = new Deferred<void>();
//TO DO: verify if value's format is wrong, and json parse failed, how to handle error
const rmMetaList: RemoteMachineMeta[] = <RemoteMachineMeta[]>JSON.parse(machineList);
let connectedRMNum: number = 0;
rmMetaList.forEach((rmMeta: RemoteMachineMeta) => {
const conn: Client = new Client();
this.machineSSHClientMap.set(rmMeta, conn);
conn.on('ready', async () => {
await this.initRemoteMachineOnConnected(rmMeta, conn);
if (++connectedRMNum === rmMetaList.length) {
deferred.resolve();
}
}).on('error', (err: Error) => {
// SSH connection error, reject with error message
deferred.reject(new Error(err.message));
}).connect({
host: rmMeta.ip,
port: rmMeta.port,
username: rmMeta.username,
password: rmMeta.passwd
});
});
return deferred.promise;
}
private async initRemoteMachineOnConnected(rmMeta: RemoteMachineMeta, conn: Client): Promise<void> {
// Create root working directory after ssh connection is ready
//TO DO: Should we mk experiments rootDir here?
const nniRootDir: string = '/tmp/nni';
await SSHClientUtility.remoteExeCommand(`mkdir -p ${this.remoteExpRootDir}`, conn);
// Copy NNI scripts to remote expeirment working directory
const remoteScriptsDir: string = this.getRemoteScriptsPath();
await SSHClientUtility.remoteExeCommand(`mkdir -p ${remoteScriptsDir}`, conn);
await SSHClientUtility.copyDirectoryToRemote('./scripts', remoteScriptsDir, conn);
await SSHClientUtility.remoteExeCommand(`chmod 777 ${nniRootDir} ${nniRootDir}/* ${nniRootDir}/scripts/*`, conn);
//Begin to execute gpu_metrics_collection scripts
SSHClientUtility.remoteExeCommand(`cd ${remoteScriptsDir} && python3 gpu_metrics_collector.py`, conn);
this.timer.subscribe(
async (tick: number) => {
const cmdresult: RemoteCommandResult = await SSHClientUtility.remoteExeCommand(
`tail -n 1 ${path.join(remoteScriptsDir, 'gpu_metrics')}`, conn);
if (cmdresult && cmdresult.stdout) {
rmMeta.gpuSummary = <GPUSummary>JSON.parse(cmdresult.stdout);
}
}
);
}
private async prepareTrialJob(trialJobId: string): Promise<boolean> {
const deferred : Deferred<boolean> = new Deferred<boolean>();
if (!this.trialConfig) {
throw new Error('trial config is not initialized');
}
const trialJobDetail: RemoteMachineTrialJobDetail | undefined = this.trialJobsMap.get(trialJobId);
if (trialJobDetail === undefined) {
throw new NNIError(NNIErrorNames.INVALID_JOB_DETAIL, `Invalid job detail information for trial job ${trialJobId}`);
}
// get an ssh client from scheduler
const rmScheduleResult: RemoteMachineScheduleResult = this.gpuScheduler.scheduleMachine(this.trialConfig.gpuNum, trialJobId);
if (rmScheduleResult.resultType === ScheduleResultType.REQUIRE_EXCEED_TOTAL) {
const errorMessage : string = `Required GPU number ${this.trialConfig.gpuNum} is too large, no machine can meet`;
this.log.error(errorMessage);
deferred.reject();
throw new NNIError(NNIErrorNames.RESOURCE_NOT_AVAILABLE, errorMessage);
} else if(rmScheduleResult.resultType == ScheduleResultType.SUCCEED
&& rmScheduleResult.scheduleInfo !== undefined) {
const rmScheduleInfo : RemoteMachineScheduleInfo = rmScheduleResult.scheduleInfo;
const trialWorkingFolder: string = path.join(this.remoteExpRootDir, 'trials', trialJobId);
await this.launchTrialOnScheduledMachine(
trialJobId, trialWorkingFolder, <TrialJobApplicationForm>trialJobDetail.form, rmScheduleInfo);
trialJobDetail.status = 'RUNNING';
trialJobDetail.url = `file://${rmScheduleInfo.rmMeta.ip}:${trialWorkingFolder}`;
trialJobDetail.startTime = new Date();
trialJobDetail.rmMeta = rmScheduleInfo.rmMeta;
deferred.resolve(true);
} else if(rmScheduleResult.resultType == ScheduleResultType.TMP_NO_AVAILABLE_GPU) {
this.log.info(`Right now no available GPU can be allocated for trial ${trialJobId}, will try to schedule later`);
deferred.resolve(false);
} else {
deferred.reject('Invalid schedule resutl type: ' + rmScheduleResult.resultType);
}
return deferred.promise;
}
private async launchTrialOnScheduledMachine(trialJobId: string, trialWorkingFolder: string, form: TrialJobApplicationForm,
rmScheduleInfo: RemoteMachineScheduleInfo): Promise<void> {
if (!this.trialConfig) {
throw new Error('trial config is not initialized');
}
const cuda_visible_device: string = rmScheduleInfo.cuda_visible_device;
const sshClient: Client = rmScheduleInfo.client;
const trialLocalTempFolder: string = path.join(this.expRootDir, 'trials-local', trialJobId);
await SSHClientUtility.remoteExeCommand(`mkdir -p ${trialWorkingFolder}`, sshClient);
await SSHClientUtility.remoteExeCommand(`mkdir -p ${path.join(trialWorkingFolder, '.nni')}`, sshClient);
await SSHClientUtility.remoteExeCommand(`touch ${path.join(trialWorkingFolder, '.nni', 'metrics')}`, sshClient);
// RemoteMachineRunShellFormat is the run shell format string,
// See definition in remoteMachineData.ts
const runScriptContent: string = String.Format(
REMOTEMACHINERUNSHELLFORMAT,
trialWorkingFolder,
trialJobId,
path.join(trialWorkingFolder, '.nni', 'jobpid'),
// Set CUDA_VISIBLE_DEVICES environment variable based on cuda_visible_device
// If no valid cuda_visible_device is defined, set CUDA_VISIBLE_DEVICES to empty string to hide GPU device
(typeof cuda_visible_device === 'string' && cuda_visible_device.length > 0) ?
`CUDA_VISIBLE_DEVICES=${cuda_visible_device} ` : `CUDA_VISIBLE_DEVICES=" " `,
this.trialConfig.command,
path.join(trialWorkingFolder, '.nni', 'stderr'),
path.join(trialWorkingFolder, '.nni', 'code'));
//create tmp trial working folder locally.
await cpp.exec(`mkdir -p ${trialLocalTempFolder}`);
// Write file content ( run.sh and parameter.cfg ) to local tmp files
await fs.promises.writeFile(path.join(trialLocalTempFolder, 'run.sh'), runScriptContent, { encoding: 'utf8' });
await fs.promises.writeFile(path.join(trialLocalTempFolder, 'parameter.cfg'), form.hyperParameters, { encoding: 'utf8' });
// Copy local tmp files to remote machine
await SSHClientUtility.copyFileToRemote(
path.join(trialLocalTempFolder, 'run.sh'), path.join(trialWorkingFolder, 'run.sh'), sshClient);
await SSHClientUtility.copyFileToRemote(
path.join(trialLocalTempFolder, 'parameter.cfg'), path.join(trialWorkingFolder, 'parameter.cfg'), sshClient);
// Copy files in codeDir to remote working directory
await SSHClientUtility.copyDirectoryToRemote(this.trialConfig.codeDir, trialWorkingFolder, sshClient);
// Execute command in remote machine
SSHClientUtility.remoteExeCommand(`bash ${path.join(trialWorkingFolder, 'run.sh')}`, sshClient);
}
private async runHostJob(form: HostJobApplicationForm): Promise<TrialJobDetail> {
const rmMeta: RemoteMachineMeta = this.getRmMetaByHost(form.host);
const sshClient: Client | undefined = this.machineSSHClientMap.get(rmMeta);
if (sshClient === undefined) {
throw new Error('sshClient not found.');
}
const jobId: string = uniqueString(5);
const localDir: string = path.join(this.expRootDir, 'hostjobs-local', jobId);
const remoteDir: string = this.getHostJobRemoteDir(jobId);
await cpp.exec(`mkdir -p ${localDir}`);
await SSHClientUtility.remoteExeCommand(`mkdir -p ${remoteDir}`, sshClient);
const runScriptContent: string = String.Format(
HOSTJOBSHELLFORMAT, remoteDir, path.join(remoteDir, 'jobpid'), form.cmd, path.join(remoteDir, 'code')
);
await fs.promises.writeFile(path.join(localDir, 'run.sh'), runScriptContent, { encoding: 'utf8' });
await SSHClientUtility.copyFileToRemote(
path.join(localDir, 'run.sh'), path.join(remoteDir, 'run.sh'), sshClient);
SSHClientUtility.remoteExeCommand(`bash ${path.join(remoteDir, 'run.sh')}`, sshClient);
const jobDetail: RemoteMachineTrialJobDetail = new RemoteMachineTrialJobDetail(jobId, 'RUNNING', new Date(), remoteDir, form);
jobDetail.rmMeta = rmMeta;
jobDetail.startTime = new Date();
this.trialJobsMap.set(jobId, jobDetail);
this.log.debug(`runHostJob: return: ${JSON.stringify(jobDetail)} `);
return jobDetail;
}
private getRmMetaByHost(host: string): RemoteMachineMeta {
for (const [rmMeta, client] of this.machineSSHClientMap.entries()) {
if (rmMeta.ip === host) {
return rmMeta;
}
}
throw new Error(`Host not found: ${host}`);
}
private async updateTrialJobStatus(trialJob: RemoteMachineTrialJobDetail, sshClient: Client): Promise<TrialJobDetail> {
const deferred: Deferred<TrialJobDetail> = new Deferred<TrialJobDetail>();
const jobpidPath: string = this.getJobPidPath(trialJob.id);
const trialReturnCodeFilePath: string = path.join(this.remoteExpRootDir, 'trials', trialJob.id, '.nni', 'code');
try {
const killResult: number = (await SSHClientUtility.remoteExeCommand(`kill -0 \`cat ${jobpidPath}\``, sshClient)).exitCode;
// if the process of jobpid is not alive any more
if (killResult !== 0) {
const trailReturnCode: string = await SSHClientUtility.getRemoteFileContent(trialReturnCodeFilePath, sshClient);
this.log.debug(`trailjob ${trialJob.id} return code: ${trailReturnCode}`);
const match: RegExpMatchArray | null = trailReturnCode.trim().match(/^(\d+)\s+(\d+)$/);
if (match) {
const { 1: code, 2: timestamp } = match;
// Update trial job's status based on result code
if (parseInt(code, 10) === 0) {
trialJob.status = 'SUCCEEDED';
} else {
trialJob.status = 'FAILED';
}
trialJob.endTime = new Date(parseInt(timestamp, 10));
}
this.log.info(`trailJob status update: ${trialJob.id}, ${trialJob.status}`);
}
deferred.resolve(trialJob);
} catch (error) {
this.log.error(`Update job status exception, error is ${error.message}`);
if (error instanceof NNIError && error.name === NNIErrorNames.NOT_FOUND) {
deferred.resolve(trialJob);
} else {
trialJob.status = 'UNKNOWN';
deferred.resolve(trialJob);
}
}
return deferred.promise;
}
private getRemoteScriptsPath(): string {
return path.join(path.dirname(path.dirname(this.remoteExpRootDir)), 'scripts');
}
private getHostJobRemoteDir(jobId: string): string {
return path.join(this.remoteExpRootDir, 'hostjobs', jobId);
}
private getRemoteModeExperimentRootDir(): string{
return path.join(os.tmpdir(), 'nni', 'experiments', getExperimentId());
}
private getJobPidPath(jobId: string): string {
const trialJobDetail: RemoteMachineTrialJobDetail | undefined = this.trialJobsMap.get(jobId);
if (trialJobDetail === undefined) {
throw new NNIError(NNIErrorNames.INVALID_JOB_DETAIL, `Invalid job detail information for trial job ${jobId}`);
}
let jobpidPath: string;
if (trialJobDetail.form.jobType === 'TRIAL') {
jobpidPath = path.join(trialJobDetail.workingDirectory, '.nni', 'jobpid');
} else if (trialJobDetail.form.jobType === 'HOST') {
jobpidPath = path.join(this.getHostJobRemoteDir(jobId), 'jobpid');
} else {
throw new Error(`Job type not supported: ${trialJobDetail.form.jobType}`);
}
return jobpidPath;
}
}
export { RemoteMachineTrainingService };
src/nni_manager/training_service/remote_machine/sshClientUtility.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import * as cpp from 'child-process-promise';
import * as fs from 'fs';
import * as path from 'path';
import { Client, ClientChannel, SFTPWrapper } from 'ssh2';
import * as stream from "stream";
import { Deferred } from 'ts-deferred';
import { NNIError, NNIErrorNames } from '../../common/errors';
import { getExperimentRootDir } from '../../common/utils';
import { RemoteCommandResult } from './remoteMachineData';
/**
*
* Utility for frequent operations towards SSH client
*
*/
export namespace SSHClientUtility {
/**
* Copy files and directories in local directory recursively to remote directory
* @param localDirectory local diretory
* @param remoteDirectory remote directory
* @param sshClient SSH client
*/
export async function copyDirectoryToRemote(localDirectory : string, remoteDirectory : string, sshClient : Client) : Promise<void> {
const deferred: Deferred<void> = new Deferred<void>();
const localCompressedDir: string = path.join(getExperimentRootDir(), 'directory.tar.gz');
const remoteCompressedDir: string = path.join(remoteDirectory, 'directory.tar.gz');
// Compress files in local directory to experiment root directory
await cpp.exec(`tar -czf ${localCompressedDir} -C ${localDirectory} .`);
// Copy the compressed file to remoteDirectory and delete it
await copyFileToRemote(localCompressedDir, remoteCompressedDir, sshClient);
await cpp.exec(`rm ${localCompressedDir}`);
// Decompress the remote compressed file in and delete it
await remoteExeCommand(`tar -oxzf ${remoteCompressedDir} -C ${remoteDirectory}`, sshClient);
await remoteExeCommand(`rm ${remoteCompressedDir}`, sshClient);
deferred.resolve();
return deferred.promise;
}
/**
* Copy local file to remote path
* @param localFilePath the path of local file
* @param remoteFilePath the target path in remote machine
* @param sshClient SSH Client
*/
export function copyFileToRemote(localFilePath : string, remoteFilePath : string, sshClient : Client) : Promise<string> {
const deferred: Deferred<string> = new Deferred<string>();
sshClient.sftp((err : Error, sftp : SFTPWrapper) => {
if (err) {
deferred.reject();
}
sftp.fastPut(localFilePath, remoteFilePath, (fastPutErr : Error) => {
sftp.end();
if (fastPutErr) {
deferred.reject();
} else {
deferred.resolve('success');
}
});
});
return deferred.promise;
}
/**
* Execute command on remote machine
* @param command the command to execute remotely
* @param client SSH Client
*/
export function remoteExeCommand(command : string, client : Client): Promise<RemoteCommandResult> {
const deferred : Deferred<RemoteCommandResult> = new Deferred<RemoteCommandResult>();
let stdout: string = '';
let stderr: string = '';
let exitCode : number;
client.exec(command, (err : Error, channel : ClientChannel) => {
if (err) {
deferred.reject(err);
}
channel.on('data', function(data : any, dataStderr : any) {
if (dataStderr) {
stderr += data.toString();
}
else {
stdout += data.toString();
}
}).on('exit', (code, signal) => {
exitCode = code as number;
deferred.resolve({
stdout : stdout,
stderr : stderr,
exitCode : exitCode
});
});
});
return deferred.promise;
}
export function getRemoteFileContent(filePath: string, sshClient: Client): Promise<string> {
const deferred: Deferred<string> = new Deferred<string>();
sshClient.sftp((err: Error, sftp : SFTPWrapper) => {
if (err) {
deferred.reject(new Error(`SFTP error: ${err.message}`));
}
try {
const sftpStream : stream.Readable = sftp.createReadStream(filePath);
let dataBuffer: string = '';
sftpStream.on('data', (data : Buffer | string) => {
dataBuffer += data;
}).on('error', (streamErr: Error) => {
deferred.reject(new NNIError(NNIErrorNames.NOT_FOUND, streamErr.message));
}).on('end', () => {
deferred.resolve(dataBuffer);
});
} catch (error) {
deferred.reject(new Error(`SFTP error: ${error.message}`));
}
});
return deferred.promise;
}
}
src/nni_manager/training_service/test/localTrainingService.test.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
* MIT License
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import { TrainingService } from '../../common/trainingService';
import { LocalTrainingService } from '../local/localTrainingService';
import * as component from '../../common/component';
describe('Unit Test for LocalTrainingService', () => {
let trainingService: TrainingService
beforeEach(async () => {
trainingService = component.get(LocalTrainingService);
})
});
\ No newline at end of file
src/nni_manager/training_service/test/mockedTrial.py 0 → 100644
View file @ 252f36f8
import os
import time
METRICS_FILENAME = '.nni/metrics'
MAGIC = 'ME'
def sdk_send_data(data):
out_dir = os.getenv('NNI_SYS_DIR')
if not os.path.isdir(out_dir):
raise Exception('Can not find NNI_SYS_DIR: {}'.format(out_dir))
filename = os.path.join(out_dir, METRICS_FILENAME)
wrapped_data = data + '\n'
datalen = len(wrapped_data)
if datalen < 2:
return
with open(filename, 'a') as f:
f.write('ME{:06d}{}'.format(datalen, wrapped_data))
def user_code():
epochs = 20
val_acc = 0
batch_size = 32
for epoch in range(epochs):
#Training
time.sleep(1)
val_acc += 0.5
metrics = 'epoch: {}, val accuracy: {:.2f}, batch size: {}'.format(epoch, val_acc, batch_size)
sdk_send_data(metrics)
if __name__ == '__main__':
print('>>>start...')
user_code()
print('>>>end...')
src/nni_manager/training_service/test/remoteMachineTrainingService.test.ts 0 → 100644
View file @ 252f36f8
/**
* Copyright (c) Microsoft Corporation
* All rights reserved.
*
* MIT License
*
* Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
* documentation files (the "Software"), to deal in the Software without restriction, including without limitation
* the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and
* to permit persons to whom the Software is furnished to do so, subject to the following conditions:
* The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED *AS IS*, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING
* BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
* DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
'use strict';
import * as assert from 'assert';
import * as chai from 'chai';
import * as chaiAsPromised from 'chai-as-promised';
import * as fs from 'fs';
import * as tmp from 'tmp';
import * as component from '../../common/component';
import { TrialJobApplicationForm, TrialJobDetail, TrainingService } from '../../common/trainingService';
import { cleanupUnitTest, delay, prepareUnitTest } from '../../common/utils';
import { RemoteMachineMetadataKey } from '../remote_machine/remoteMachineData';
import { RemoteMachineTrainingService } from '../remote_machine/remoteMachineTrainingService';
// copy mockedTrail.py to local folder
const localCodeDir: string = tmp.dirSync().name
const mockedTrialPath: string = './training_service/test/mockedTrial.py'
fs.copyFileSync(mockedTrialPath, localCodeDir + '/mockedTrial.py')
describe('Unit Test for RemoteMachineTrainingService', () => {
/*
To enable remote machine unit test, remote machine information needs to be configured in:
Default/.vscode/rminfo.json, whose content looks like:
{
"ip": "10.172.121.40",
"user": "user1",
"password": "mypassword"
}
*/
let skip: boolean = false;
let testRmInfo: any;
let machineList: any;
try {
testRmInfo = JSON.parse(fs.readFileSync('../../.vscode/rminfo.json', 'utf8'));
console.log(testRmInfo);
machineList = `[{\"ip\":\"${testRmInfo.ip}\",\"port\":22,\"username\":\"${testRmInfo.user}\",\"passwd\":\"${testRmInfo.password}\"}]`;
} catch (err) {
console.log('Please configure rminfo.json to enable remote machine unit test.');
skip = true;
}
let remoteMachineTrainingService: RemoteMachineTrainingService
before(() => {
chai.should();
chai.use(chaiAsPromised);
prepareUnitTest();
});
after(() => {
cleanupUnitTest();
});
beforeEach(() => {
if (skip) {
return;
}
remoteMachineTrainingService = component.get(RemoteMachineTrainingService);
remoteMachineTrainingService.run();
});
afterEach(() => {
if (skip) {
return;
}
remoteMachineTrainingService.cleanUp();
});
it('List trial jobs', async () => {
if (skip) {
return;
}
chai.expect(await remoteMachineTrainingService.listTrialJobs()).to.be.empty;
});
it('Set cluster metadata', async () => {
if (skip) {
return;
}
await remoteMachineTrainingService.setClusterMetadata(RemoteMachineMetadataKey.MACHINE_LIST, machineList);
await remoteMachineTrainingService.setClusterMetadata(
RemoteMachineMetadataKey.TRIAL_CONFIG, `{"command":"sleep 1h && echo ","codeDir":"${localCodeDir}","gpuNum":1}`);
const form: TrialJobApplicationForm = {
jobType: 'TRIAL',
hyperParameters: 'mock hyperparameters'
};
const trialJob = await remoteMachineTrainingService.submitTrialJob(form);
// After a job is cancelled, the status should be changed to 'USER_CANCELED'
await remoteMachineTrainingService.cancelTrialJob(trialJob.id);
// After a job is cancelled, the status should be changed to 'USER_CANCELED'
const trialJob2 = await remoteMachineTrainingService.getTrialJob(trialJob.id);
chai.expect(trialJob2.status).to.be.equals('USER_CANCELED');
//Expect rejected if passing invalid trial job id
await remoteMachineTrainingService.cancelTrialJob(trialJob.id + 'ddd').should.eventually.be.rejected;
});
it('Submit job test', async () => {
if (skip) {
return;
}
});
it('Submit job and read metrics data', async () => {
if (skip) {
return;
}
// set machine list'
await remoteMachineTrainingService.setClusterMetadata(RemoteMachineMetadataKey.MACHINE_LIST, machineList);
// set meta data
const trialConfig: string = `{\"command\":\"python3 mockedTrial.py\", \"codeDir\":\"${localCodeDir}\",\"gpuNum\":0}`
await remoteMachineTrainingService.setClusterMetadata(RemoteMachineMetadataKey.TRIAL_CONFIG, trialConfig);
// submit job
const form: TrialJobApplicationForm = {
jobType: 'TRIAL',
hyperParameters: 'mock hyperparameters'
};
const jobDetail: TrialJobDetail = await remoteMachineTrainingService.submitTrialJob(form);
// Add metrics listeners
const listener1 = function f1(metric: any) {
}
const listener2 = function f1(metric: any) {
}
remoteMachineTrainingService.addTrialJobMetricListener(listener1);
remoteMachineTrainingService.addTrialJobMetricListener(listener2);
await delay(10000);
// remove listender1
remoteMachineTrainingService.removeTrialJobMetricListener(listener1);
await delay(5000);
}).timeout(30000);
it('Test getTrialJob exception', async () => {
if (skip) {
return;
}
await remoteMachineTrainingService.getTrialJob('wrongid').catch((err) => {
assert(err !== undefined);
});
});
});
src/nni_manager/tsconfig.json 0 → 100644
View file @ 252f36f8
{
"compilerOptions": {
"target": "ESNext",
"module": "CommonJS",
"removeComments": true,
"strict": true,
"noImplicitAny": true,
"baseUrl": ".",
"outDir": "dist",
"experimentalDecorators": true,
"emitDecoratorMetadata": true,
"typeRoots": [
"node_modules/@types",
"types"
]
},
"exclude": [
"node_modules"
]
}
\ No newline at end of file
src/nni_manager/tslint.json 0 → 100644
View file @ 252f36f8
{
"defaultSeverity": "error",
"extends": "tslint-microsoft-contrib",
"jsRules": {},
"rules": {
"no-relative-imports": false,
"export-name": false,
"interface-name": [true, "never-prefix"],
"no-increment-decrement": false,
"promise-function-async": false,
"no-console": [true, "log"],
"no-multiline-string": false
},
"rulesDirectory": []
}
\ No newline at end of file
src/nni_manager/types/child-process-promise/index.d.ts 0 → 100644
View file @ 252f36f8
declare module 'child-process-promise' {
export function exec(command: string): Promise<void>;
}
\ No newline at end of file
src/nni_manager/types/node-nvidia-smi/index.d.ts 0 → 100644
View file @ 252f36f8
declare module 'node-nvidia-smi' {
function smi(callback: (error: Error, data: smi.GPUInfo) => void): void;
namespace smi {
interface GPUInfo {
nvidia_smi_log: {
attached_gpus: string;
gpu: {
minor_number: string;
utilization: {
gpu_util: string;
memory_util: string;
};
process: string | object;
}[];
};
}
}
export = smi;
}
\ No newline at end of file
src/nni_manager/types/tail-stream/index.d.ts 0 → 100644
View file @ 252f36f8
declare module 'tail-stream' {
export interface Stream {
on(type: 'data', callback: (data: Buffer) => void): void;
}
export function createReadStream(path: string): Stream;
}
\ No newline at end of file
src/nni_manager/yarn.lock 0 → 100644
View file @ 252f36f8
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
"@types/body-parser@*":
version "1.17.0"
resolved "https://registry.yarnpkg.com/@types/body-parser/-/body-parser-1.17.0.tgz#9f5c9d9bd04bb54be32d5eb9fc0d8c974e6cf58c"
dependencies:
"@types/connect" "*"
"@types/node" "*"
"@types/caseless@*":
version "0.12.1"
resolved "https://registry.yarnpkg.com/@types/caseless/-/caseless-0.12.1.tgz#9794c69c8385d0192acc471a540d1f8e0d16218a"
"@types/chai-as-promised@^7.1.0":
version "7.1.0"
resolved "https://registry.yarnpkg.com/@types/chai-as-promised/-/chai-as-promised-7.1.0.tgz#010b04cde78eacfb6e72bfddb3e58fe23c2e78b9"
dependencies:
"@types/chai" "*"
"@types/chai@*", "@types/chai@^4.1.4":
version "4.1.4"
resolved "https://registry.yarnpkg.com/@types/chai/-/chai-4.1.4.tgz#5ca073b330d90b4066d6ce18f60d57f2084ce8ca"
"@types/connect@*":
version "3.4.32"
resolved "https://registry.yarnpkg.com/@types/connect/-/connect-3.4.32.tgz#aa0e9616b9435ccad02bc52b5b454ffc2c70ba28"
dependencies:
"@types/node" "*"
"@types/events@*":
version "1.2.0"
resolved "https://registry.yarnpkg.com/@types/events/-/events-1.2.0.tgz#81a6731ce4df43619e5c8c945383b3e62a89ea86"
"@types/express-serve-static-core@*":
version "4.16.0"
resolved "https://registry.yarnpkg.com/@types/express-serve-static-core/-/express-serve-static-core-4.16.0.tgz#fdfe777594ddc1fe8eb8eccce52e261b496e43e7"
dependencies:
"@types/events" "*"
"@types/node" "*"
"@types/range-parser" "*"
"@types/express@^4.16.0":
version "4.16.0"
resolved "https://registry.yarnpkg.com/@types/express/-/express-4.16.0.tgz#6d8bc42ccaa6f35cf29a2b7c3333cb47b5a32a19"
dependencies:
"@types/body-parser" "*"
"@types/express-serve-static-core" "*"
"@types/serve-static" "*"
"@types/form-data@*":
version "2.2.1"
resolved "https://registry.yarnpkg.com/@types/form-data/-/form-data-2.2.1.tgz#ee2b3b8eaa11c0938289953606b745b738c54b1e"
dependencies:
"@types/node" "*"
"@types/mime@*":
version "2.0.0"
resolved "https://registry.yarnpkg.com/@types/mime/-/mime-2.0.0.tgz#5a7306e367c539b9f6543499de8dd519fac37a8b"
"@types/mocha@^5.2.5":
version "5.2.5"
resolved "https://registry.yarnpkg.com/@types/mocha/-/mocha-5.2.5.tgz#8a4accfc403c124a0bafe8a9fc61a05ec1032073"
"@types/node@*":
version "10.5.2"
resolved "https://registry.yarnpkg.com/@types/node/-/node-10.5.2.tgz#f19f05314d5421fe37e74153254201a7bf00a707"
"@types/node@^10.5.5":
version "10.5.5"
resolved "https://registry.yarnpkg.com/@types/node/-/node-10.5.5.tgz#8e84d24e896cd77b0d4f73df274027e3149ec2ba"
"@types/range-parser@*":
version "1.2.2"
resolved "https://registry.yarnpkg.com/@types/range-parser/-/range-parser-1.2.2.tgz#fa8e1ad1d474688a757140c91de6dace6f4abc8d"
"@types/request@^2.47.1":
version "2.47.1"
resolved "https://registry.yarnpkg.com/@types/request/-/request-2.47.1.tgz#25410d3afbdac04c91a94ad9efc9824100735824"
dependencies:
"@types/caseless" "*"
"@types/form-data" "*"
"@types/node" "*"
"@types/tough-cookie" "*"
"@types/rx-core-binding@*":
version "4.0.4"
resolved "https://registry.yarnpkg.com/@types/rx-core-binding/-/rx-core-binding-4.0.4.tgz#d969d32f15a62b89e2862c17b3ee78fe329818d3"
dependencies:
"@types/rx-core" "*"
"@types/rx-core@*":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@types/rx-core/-/rx-core-4.0.3.tgz#0b3354b1238cedbe2b74f6326f139dbc7a591d60"
"@types/rx-lite-aggregates@*":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@types/rx-lite-aggregates/-/rx-lite-aggregates-4.0.3.tgz#6efb2b7f3d5f07183a1cb2bd4b1371d7073384c2"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite-async@*":
version "4.0.2"
resolved "https://registry.yarnpkg.com/@types/rx-lite-async/-/rx-lite-async-4.0.2.tgz#27fbf0caeff029f41e2d2aae638b05e91ceb600c"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite-backpressure@*":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@types/rx-lite-backpressure/-/rx-lite-backpressure-4.0.3.tgz#05abb19bdf87cc740196c355e5d0b37bb50b5d56"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite-coincidence@*":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@types/rx-lite-coincidence/-/rx-lite-coincidence-4.0.3.tgz#80bd69acc4054a15cdc1638e2dc8843498cd85c0"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite-experimental@*":
version "4.0.1"
resolved "https://registry.yarnpkg.com/@types/rx-lite-experimental/-/rx-lite-experimental-4.0.1.tgz#c532f5cbdf3f2c15da16ded8930d1b2984023cbd"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite-joinpatterns@*":
version "4.0.1"
resolved "https://registry.yarnpkg.com/@types/rx-lite-joinpatterns/-/rx-lite-joinpatterns-4.0.1.tgz#f70fe370518a8432f29158cc92ffb56b4e4afc3e"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite-testing@*":
version "4.0.1"
resolved "https://registry.yarnpkg.com/@types/rx-lite-testing/-/rx-lite-testing-4.0.1.tgz#21b19d11f4dfd6ffef5a9d1648e9c8879bfe21e9"
dependencies:
"@types/rx-lite-virtualtime" "*"
"@types/rx-lite-time@*":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@types/rx-lite-time/-/rx-lite-time-4.0.3.tgz#0eda65474570237598f3448b845d2696f2dbb1c4"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite-virtualtime@*":
version "4.0.3"
resolved "https://registry.yarnpkg.com/@types/rx-lite-virtualtime/-/rx-lite-virtualtime-4.0.3.tgz#4b30cacd0fe2e53af29f04f7438584c7d3959537"
dependencies:
"@types/rx-lite" "*"
"@types/rx-lite@*":
version "4.0.5"
resolved "https://registry.yarnpkg.com/@types/rx-lite/-/rx-lite-4.0.5.tgz#b3581525dff69423798daa9a0d33c1e66a5e8c4c"
dependencies:
"@types/rx-core" "*"
"@types/rx-core-binding" "*"
"@types/rx@^4.1.1":
version "4.1.1"
resolved "https://registry.yarnpkg.com/@types/rx/-/rx-4.1.1.tgz#598fc94a56baed975f194574e0f572fd8e627a48"
dependencies:
"@types/rx-core" "*"
"@types/rx-core-binding" "*"
"@types/rx-lite" "*"
"@types/rx-lite-aggregates" "*"
"@types/rx-lite-async" "*"
"@types/rx-lite-backpressure" "*"
"@types/rx-lite-coincidence" "*"
"@types/rx-lite-experimental" "*"
"@types/rx-lite-joinpatterns" "*"
"@types/rx-lite-testing" "*"
"@types/rx-lite-time" "*"
"@types/rx-lite-virtualtime" "*"
"@types/serve-static@*":
version "1.13.2"
resolved "https://registry.yarnpkg.com/@types/serve-static/-/serve-static-1.13.2.tgz#f5ac4d7a6420a99a6a45af4719f4dcd8cd907a48"
dependencies:
"@types/express-serve-static-core" "*"
"@types/mime" "*"
"@types/sqlite3@^3.1.3":
version "3.1.3"
resolved "https://registry.yarnpkg.com/@types/sqlite3/-/sqlite3-3.1.3.tgz#580d547203b8ad6e11aa6a6769c8ca5d7e197d13"
dependencies:
"@types/events" "*"
"@types/node" "*"
"@types/ssh2-streams@*":
version "0.1.2"
resolved "https://registry.yarnpkg.com/@types/ssh2-streams/-/ssh2-streams-0.1.2.tgz#7aa18b8c2450f17699e9ea18a76efc838188d58d"
dependencies:
"@types/node" "*"
"@types/ssh2@^0.5.35":
version "0.5.35"
resolved "https://registry.yarnpkg.com/@types/ssh2/-/ssh2-0.5.35.tgz#d6e60d59b7fc22db10abf4730aa7448babde7e3b"
dependencies:
"@types/node" "*"
"@types/ssh2-streams" "*"
"@types/stream-buffers@^3.0.2":
version "3.0.2"
resolved "https://registry.yarnpkg.com/@types/stream-buffers/-/stream-buffers-3.0.2.tgz#b73bfcceae39ecb259750b44ef38a36cfc20e370"
dependencies:
"@types/node" "*"
"@types/tmp@^0.0.33":
version "0.0.33"
resolved "https://registry.yarnpkg.com/@types/tmp/-/tmp-0.0.33.tgz#1073c4bc824754ae3d10cfab88ab0237ba964e4d"
"@types/tough-cookie@*":
version "2.3.3"
resolved "https://registry.yarnpkg.com/@types/tough-cookie/-/tough-cookie-2.3.3.tgz#7f226d67d654ec9070e755f46daebf014628e9d9"
"@zeit/schemas@2.1.1":
version "2.1.1"
resolved "https://registry.yarnpkg.com/@zeit/schemas/-/schemas-2.1.1.tgz#bca9d84df177c85f2d2a7dad37512f384761b23d"
abbrev@1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/abbrev/-/abbrev-1.1.1.tgz#f8f2c887ad10bf67f634f005b6987fed3179aac8"
accepts@~1.3.5:
version "1.3.5"
resolved "https://registry.yarnpkg.com/accepts/-/accepts-1.3.5.tgz#eb777df6011723a3b14e8a72c0805c8e86746bd2"
dependencies:
mime-types "~2.1.18"
negotiator "0.6.1"
ajv@6.5.2:
version "6.5.2"
resolved "https://registry.yarnpkg.com/ajv/-/ajv-6.5.2.tgz#678495f9b82f7cca6be248dd92f59bff5e1f4360"
dependencies:
fast-deep-equal "^2.0.1"
fast-json-stable-stringify "^2.0.0"
json-schema-traverse "^0.4.1"
uri-js "^4.2.1"
ajv@^5.1.0:
version "5.5.2"
resolved "https://registry.yarnpkg.com/ajv/-/ajv-5.5.2.tgz#73b5eeca3fab653e3d3f9422b341ad42205dc965"
dependencies:
co "^4.6.0"
fast-deep-equal "^1.0.0"
fast-json-stable-stringify "^2.0.0"
json-schema-traverse "^0.3.0"
ansi-align@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/ansi-align/-/ansi-align-2.0.0.tgz#c36aeccba563b89ceb556f3690f0b1d9e3547f7f"
dependencies:
string-width "^2.0.0"
ansi-regex@^2.0.0:
version "2.1.1"
resolved "https://registry.yarnpkg.com/ansi-regex/-/ansi-regex-2.1.1.tgz#c3b33ab5ee360d86e0e628f0468ae7ef27d654df"
ansi-regex@^3.0.0:
version "3.0.0"
resolved "https://registry.yarnpkg.com/ansi-regex/-/ansi-regex-3.0.0.tgz#ed0317c322064f79466c02966bddb605ab37d998"
ansi-styles@^2.2.1:
version "2.2.1"
resolved "https://registry.yarnpkg.com/ansi-styles/-/ansi-styles-2.2.1.tgz#b432dd3358b634cf75e1e4664368240533c1ddbe"
ansi-styles@^3.2.1:
version "3.2.1"
resolved "https://registry.yarnpkg.com/ansi-styles/-/ansi-styles-3.2.1.tgz#41fbb20243e50b12be0f04b8dedbf07520ce841d"
dependencies:
color-convert "^1.9.0"
aproba@^1.0.3:
version "1.2.0"
resolved "https://registry.yarnpkg.com/aproba/-/aproba-1.2.0.tgz#6802e6264efd18c790a1b0d517f0f2627bf2c94a"
arch@^2.1.0:
version "2.1.1"
resolved "https://registry.yarnpkg.com/arch/-/arch-2.1.1.tgz#8f5c2731aa35a30929221bb0640eed65175ec84e"
are-we-there-yet@~1.1.2:
version "1.1.5"
resolved "https://registry.yarnpkg.com/are-we-there-yet/-/are-we-there-yet-1.1.5.tgz#4b35c2944f062a8bfcda66410760350fe9ddfc21"
dependencies:
delegates "^1.0.0"
readable-stream "^2.0.6"
arg@2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/arg/-/arg-2.0.0.tgz#c06e7ff69ab05b3a4a03ebe0407fac4cba657545"
argparse@^1.0.7:
version "1.0.10"
resolved "https://registry.yarnpkg.com/argparse/-/argparse-1.0.10.tgz#bcd6791ea5ae09725e17e5ad988134cd40b3d911"
dependencies:
sprintf-js "~1.0.2"
array-flatten@1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/array-flatten/-/array-flatten-1.1.1.tgz#9a5f699051b1e7073328f2a008968b64ea2955d2"
array-union@^1.0.1:
version "1.0.2"
resolved "https://registry.yarnpkg.com/array-union/-/array-union-1.0.2.tgz#9a34410e4f4e3da23dea375be5be70f24778ec39"
dependencies:
array-uniq "^1.0.1"
array-uniq@^1.0.1:
version "1.0.3"
resolved "https://registry.yarnpkg.com/array-uniq/-/array-uniq-1.0.3.tgz#af6ac877a25cc7f74e058894753858dfdb24fdb6"
arrify@^1.0.0:
version "1.0.1"
resolved "https://registry.yarnpkg.com/arrify/-/arrify-1.0.1.tgz#898508da2226f380df904728456849c1501a4b0d"
asn1@~0.2.0, asn1@~0.2.3:
version "0.2.3"
resolved "https://registry.yarnpkg.com/asn1/-/asn1-0.2.3.tgz#dac8787713c9966849fc8180777ebe9c1ddf3b86"
assert-plus@1.0.0, assert-plus@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/assert-plus/-/assert-plus-1.0.0.tgz#f12e0f3c5d77b0b1cdd9146942e4e96c1e4dd525"
assertion-error@^1.0.1:
version "1.1.0"
resolved "https://registry.yarnpkg.com/assertion-error/-/assertion-error-1.1.0.tgz#e60b6b0e8f301bd97e5375215bda406c85118c0b"
asynckit@^0.4.0:
version "0.4.0"
resolved "https://registry.yarnpkg.com/asynckit/-/asynckit-0.4.0.tgz#c79ed97f7f34cb8f2ba1bc9790bcc366474b4b79"
aws-sign2@~0.7.0:
version "0.7.0"
resolved "https://registry.yarnpkg.com/aws-sign2/-/aws-sign2-0.7.0.tgz#b46e890934a9591f2d2f6f86d7e6a9f1b3fe76a8"
aws4@^1.6.0:
version "1.7.0"
resolved "https://registry.yarnpkg.com/aws4/-/aws4-1.7.0.tgz#d4d0e9b9dbfca77bf08eeb0a8a471550fe39e289"
babel-code-frame@^6.22.0:
version "6.26.0"
resolved "https://registry.yarnpkg.com/babel-code-frame/-/babel-code-frame-6.26.0.tgz#63fd43f7dc1e3bb7ce35947db8fe369a3f58c74b"
dependencies:
chalk "^1.1.3"
esutils "^2.0.2"
js-tokens "^3.0.2"
balanced-match@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/balanced-match/-/balanced-match-1.0.0.tgz#89b4d199ab2bee49de164ea02b89ce462d71b767"
bcrypt-pbkdf@^1.0.0:
version "1.0.2"
resolved "https://registry.yarnpkg.com/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.2.tgz#a4301d389b6a43f9b67ff3ca11a3f6637e360e9e"
dependencies:
tweetnacl "^0.14.3"
body-parser@1.18.2:
version "1.18.2"
resolved "https://registry.yarnpkg.com/body-parser/-/body-parser-1.18.2.tgz#87678a19d84b47d859b83199bd59bce222b10454"
dependencies:
bytes "3.0.0"
content-type "~1.0.4"
debug "2.6.9"
depd "~1.1.1"
http-errors "~1.6.2"
iconv-lite "0.4.19"
on-finished "~2.3.0"
qs "6.5.1"
raw-body "2.3.2"
type-is "~1.6.15"
boxen@1.3.0:
version "1.3.0"
resolved "https://registry.yarnpkg.com/boxen/-/boxen-1.3.0.tgz#55c6c39a8ba58d9c61ad22cd877532deb665a20b"
dependencies:
ansi-align "^2.0.0"
camelcase "^4.0.0"
chalk "^2.0.1"
cli-boxes "^1.0.0"
string-width "^2.0.0"
term-size "^1.2.0"
widest-line "^2.0.0"
brace-expansion@^1.1.7:
version "1.1.11"
resolved "https://registry.yarnpkg.com/brace-expansion/-/brace-expansion-1.1.11.tgz#3c7fcbf529d87226f3d2f52b966ff5271eb441dd"
dependencies:
balanced-match "^1.0.0"
concat-map "0.0.1"
browser-stdout@1.3.1:
version "1.3.1"
resolved "https://registry.yarnpkg.com/browser-stdout/-/browser-stdout-1.3.1.tgz#baa559ee14ced73452229bad7326467c61fabd60"
buffer-from@^1.0.0, buffer-from@^1.1.0:
version "1.1.1"
resolved "https://registry.yarnpkg.com/buffer-from/-/buffer-from-1.1.1.tgz#32713bc028f75c02fdb710d7c7bcec1f2c6070ef"
builtin-modules@^1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/builtin-modules/-/builtin-modules-1.1.1.tgz#270f076c5a72c02f5b65a47df94c5fe3a278892f"
bytes@3.0.0:
version "3.0.0"
resolved "https://registry.yarnpkg.com/bytes/-/bytes-3.0.0.tgz#d32815404d689699f85a4ea4fa8755dd13a96048"
callsites@^1.0.0:
version "1.0.1"
resolved "https://registry.yarnpkg.com/callsites/-/callsites-1.0.1.tgz#c14c24188ce8e1d6a030b4c3c942e6ba895b6a1a"
camelcase@^4.0.0:
version "4.1.0"
resolved "https://registry.yarnpkg.com/camelcase/-/camelcase-4.1.0.tgz#d545635be1e33c542649c69173e5de6acfae34dd"
caseless@~0.12.0:
version "0.12.0"
resolved "https://registry.yarnpkg.com/caseless/-/caseless-0.12.0.tgz#1b681c21ff84033c826543090689420d187151dc"
chai-as-promised@^7.1.1:
version "7.1.1"
resolved "https://registry.yarnpkg.com/chai-as-promised/-/chai-as-promised-7.1.1.tgz#08645d825deb8696ee61725dbf590c012eb00ca0"
dependencies:
check-error "^1.0.2"
chai@^4.1.2:
version "4.1.2"
resolved "https://registry.yarnpkg.com/chai/-/chai-4.1.2.tgz#0f64584ba642f0f2ace2806279f4f06ca23ad73c"
dependencies:
assertion-error "^1.0.1"
check-error "^1.0.1"
deep-eql "^3.0.0"
get-func-name "^2.0.0"
pathval "^1.0.0"
type-detect "^4.0.0"
chalk@2.4.1, chalk@^2.0.1, chalk@^2.3.0:
version "2.4.1"
resolved "https://registry.yarnpkg.com/chalk/-/chalk-2.4.1.tgz#18c49ab16a037b6eb0152cc83e3471338215b66e"
dependencies:
ansi-styles "^3.2.1"
escape-string-regexp "^1.0.5"
supports-color "^5.3.0"
chalk@^1.1.3:
version "1.1.3"
resolved "https://registry.yarnpkg.com/chalk/-/chalk-1.1.3.tgz#a8115c55e4a702fe4d150abd3872822a7e09fc98"
dependencies:
ansi-styles "^2.2.1"
escape-string-regexp "^1.0.2"
has-ansi "^2.0.0"
strip-ansi "^3.0.0"
supports-color "^2.0.0"
check-error@^1.0.1, check-error@^1.0.2:
version "1.0.2"
resolved "https://registry.yarnpkg.com/check-error/-/check-error-1.0.2.tgz#574d312edd88bb5dd8912e9286dd6c0aed4aac82"
child-process-promise@^2.2.1:
version "2.2.1"
resolved "https://registry.yarnpkg.com/child-process-promise/-/child-process-promise-2.2.1.tgz#4730a11ef610fad450b8f223c79d31d7bdad8074"
dependencies:
cross-spawn "^4.0.2"
node-version "^1.0.0"
promise-polyfill "^6.0.1"
chownr@^1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/chownr/-/chownr-1.0.1.tgz#e2a75042a9551908bebd25b8523d5f9769d79181"
cli-boxes@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/cli-boxes/-/cli-boxes-1.0.0.tgz#4fa917c3e59c94a004cd61f8ee509da651687143"
clipboardy@1.2.3:
version "1.2.3"
resolved "https://registry.yarnpkg.com/clipboardy/-/clipboardy-1.2.3.tgz#0526361bf78724c1f20be248d428e365433c07ef"
dependencies:
arch "^2.1.0"
execa "^0.8.0"
co@^4.6.0:
version "4.6.0"
resolved "https://registry.yarnpkg.com/co/-/co-4.6.0.tgz#6ea6bdf3d853ae54ccb8e47bfa0bf3f9031fb184"
code-point-at@^1.0.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/code-point-at/-/code-point-at-1.1.0.tgz#0d070b4d043a5bea33a2f1a40e2edb3d9a4ccf77"
color-convert@^1.9.0:
version "1.9.2"
resolved "https://registry.yarnpkg.com/color-convert/-/color-convert-1.9.2.tgz#49881b8fba67df12a96bdf3f56c0aab9e7913147"
dependencies:
color-name "1.1.1"
color-name@1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/color-name/-/color-name-1.1.1.tgz#4b1415304cf50028ea81643643bd82ea05803689"
combined-stream@1.0.6, combined-stream@~1.0.5:
version "1.0.6"
resolved "https://registry.yarnpkg.com/combined-stream/-/combined-stream-1.0.6.tgz#723e7df6e801ac5613113a7e445a9b69cb632818"
dependencies:
delayed-stream "~1.0.0"
commander@2.15.1:
version "2.15.1"
resolved "https://registry.yarnpkg.com/commander/-/commander-2.15.1.tgz#df46e867d0fc2aec66a34662b406a9ccafff5b0f"
commander@^2.12.1:
version "2.16.0"
resolved "https://registry.yarnpkg.com/commander/-/commander-2.16.0.tgz#f16390593996ceb4f3eeb020b31d78528f7f8a50"
concat-map@0.0.1:
version "0.0.1"
resolved "https://registry.yarnpkg.com/concat-map/-/concat-map-0.0.1.tgz#d8a96bd77fd68df7793a73036a3ba0d5405d477b"
console-control-strings@^1.0.0, console-control-strings@~1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/console-control-strings/-/console-control-strings-1.1.0.tgz#3d7cf4464db6446ea644bf4b39507f9851008e8e"
content-disposition@0.5.2:
version "0.5.2"
resolved "https://registry.yarnpkg.com/content-disposition/-/content-disposition-0.5.2.tgz#0cf68bb9ddf5f2be7961c3a85178cb85dba78cb4"
content-type@~1.0.4:
version "1.0.4"
resolved "https://registry.yarnpkg.com/content-type/-/content-type-1.0.4.tgz#e138cc75e040c727b1966fe5e5f8c9aee256fe3b"
cookie-signature@1.0.6:
version "1.0.6"
resolved "https://registry.yarnpkg.com/cookie-signature/-/cookie-signature-1.0.6.tgz#e303a882b342cc3ee8ca513a79999734dab3ae2c"
cookie@0.3.1:
version "0.3.1"
resolved "https://registry.yarnpkg.com/cookie/-/cookie-0.3.1.tgz#e7e0a1f9ef43b4c8ba925c5c5a96e806d16873bb"
core-util-is@1.0.2, core-util-is@~1.0.0:
version "1.0.2"
resolved "https://registry.yarnpkg.com/core-util-is/-/core-util-is-1.0.2.tgz#b5fd54220aa2bc5ab57aab7140c940754503c1a7"
cross-spawn@^4.0.2:
version "4.0.2"
resolved "https://registry.yarnpkg.com/cross-spawn/-/cross-spawn-4.0.2.tgz#7b9247621c23adfdd3856004a823cbe397424d41"
dependencies:
lru-cache "^4.0.1"
which "^1.2.9"
cross-spawn@^5.0.1:
version "5.1.0"
resolved "https://registry.yarnpkg.com/cross-spawn/-/cross-spawn-5.1.0.tgz#e8bd0efee58fcff6f8f94510a0a554bbfa235449"
dependencies:
lru-cache "^4.0.1"
shebang-command "^1.2.0"
which "^1.2.9"
dashdash@^1.12.0:
version "1.14.1"
resolved "https://registry.yarnpkg.com/dashdash/-/dashdash-1.14.1.tgz#853cfa0f7cbe2fed5de20326b8dd581035f6e2f0"
dependencies:
assert-plus "^1.0.0"
debug@2.6.9, debug@^2.1.2:
version "2.6.9"
resolved "https://registry.yarnpkg.com/debug/-/debug-2.6.9.tgz#5d128515df134ff327e90a4c93f4e077a536341f"
dependencies:
ms "2.0.0"
debug@3.1.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/debug/-/debug-3.1.0.tgz#5bb5a0672628b64149566ba16819e61518c67261"
dependencies:
ms "2.0.0"
deep-eql@^3.0.0:
version "3.0.1"
resolved "https://registry.yarnpkg.com/deep-eql/-/deep-eql-3.0.1.tgz#dfc9404400ad1c8fe023e7da1df1c147c4b444df"
dependencies:
type-detect "^4.0.0"
deep-extend@^0.6.0:
version "0.6.0"
resolved "https://registry.yarnpkg.com/deep-extend/-/deep-extend-0.6.0.tgz#c4fa7c95404a17a9c3e8ca7e1537312b736330ac"
delayed-stream@~1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/delayed-stream/-/delayed-stream-1.0.0.tgz#df3ae199acadfb7d440aaae0b29e2272b24ec619"
delegates@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/delegates/-/delegates-1.0.0.tgz#84c6e159b81904fdca59a0ef44cd870d31250f9a"
depd@1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/depd/-/depd-1.1.1.tgz#5783b4e1c459f06fa5ca27f991f3d06e7a310359"
depd@~1.1.1, depd@~1.1.2:
version "1.1.2"
resolved "https://registry.yarnpkg.com/depd/-/depd-1.1.2.tgz#9bcd52e14c097763e749b274c4346ed2e560b5a9"
destroy@~1.0.4:
version "1.0.4"
resolved "https://registry.yarnpkg.com/destroy/-/destroy-1.0.4.tgz#978857442c44749e4206613e37946205826abd80"
detect-libc@^1.0.2:
version "1.0.3"
resolved "https://registry.yarnpkg.com/detect-libc/-/detect-libc-1.0.3.tgz#fa137c4bd698edf55cd5cd02ac559f91a4c4ba9b"
diff@3.5.0, diff@^3.1.0, diff@^3.2.0:
version "3.5.0"
resolved "https://registry.yarnpkg.com/diff/-/diff-3.5.0.tgz#800c0dd1e0a8bfbc95835c202ad220fe317e5a12"
ecc-jsbn@~0.1.1:
version "0.1.1"
resolved "https://registry.yarnpkg.com/ecc-jsbn/-/ecc-jsbn-0.1.1.tgz#0fc73a9ed5f0d53c38193398523ef7e543777505"
dependencies:
jsbn "~0.1.0"
ee-first@1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/ee-first/-/ee-first-1.1.1.tgz#590c61156b0ae2f4f0255732a158b266bc56b21d"
encodeurl@~1.0.2:
version "1.0.2"
resolved "https://registry.yarnpkg.com/encodeurl/-/encodeurl-1.0.2.tgz#ad3ff4c86ec2d029322f5a02c3a9a606c95b3f59"
escape-html@~1.0.3:
version "1.0.3"
resolved "https://registry.yarnpkg.com/escape-html/-/escape-html-1.0.3.tgz#0258eae4d3d0c0974de1c169188ef0051d1d1988"
escape-string-regexp@1.0.5, escape-string-regexp@^1.0.2, escape-string-regexp@^1.0.5:
version "1.0.5"
resolved "https://registry.yarnpkg.com/escape-string-regexp/-/escape-string-regexp-1.0.5.tgz#1b61c0562190a8dff6ae3bb2cf0200ca130b86d4"
esprima@^4.0.0:
version "4.0.1"
resolved "https://registry.yarnpkg.com/esprima/-/esprima-4.0.1.tgz#13b04cdb3e6c5d19df91ab6987a8695619b0aa71"
esutils@^2.0.2:
version "2.0.2"
resolved "https://registry.yarnpkg.com/esutils/-/esutils-2.0.2.tgz#0abf4f1caa5bcb1f7a9d8acc6dea4faaa04bac9b"
etag@~1.8.1:
version "1.8.1"
resolved "https://registry.yarnpkg.com/etag/-/etag-1.8.1.tgz#41ae2eeb65efa62268aebfea83ac7d79299b0887"
execa@^0.7.0:
version "0.7.0"
resolved "https://registry.yarnpkg.com/execa/-/execa-0.7.0.tgz#944becd34cc41ee32a63a9faf27ad5a65fc59777"
dependencies:
cross-spawn "^5.0.1"
get-stream "^3.0.0"
is-stream "^1.1.0"
npm-run-path "^2.0.0"
p-finally "^1.0.0"
signal-exit "^3.0.0"
strip-eof "^1.0.0"
execa@^0.8.0:
version "0.8.0"
resolved "https://registry.yarnpkg.com/execa/-/execa-0.8.0.tgz#d8d76bbc1b55217ed190fd6dd49d3c774ecfc8da"
dependencies:
cross-spawn "^5.0.1"
get-stream "^3.0.0"
is-stream "^1.1.0"
npm-run-path "^2.0.0"
p-finally "^1.0.0"
signal-exit "^3.0.0"
strip-eof "^1.0.0"
express@^4.16.3:
version "4.16.3"
resolved "https://registry.yarnpkg.com/express/-/express-4.16.3.tgz#6af8a502350db3246ecc4becf6b5a34d22f7ed53"
dependencies:
accepts "~1.3.5"
array-flatten "1.1.1"
body-parser "1.18.2"
content-disposition "0.5.2"
content-type "~1.0.4"
cookie "0.3.1"
cookie-signature "1.0.6"
debug "2.6.9"
depd "~1.1.2"
encodeurl "~1.0.2"
escape-html "~1.0.3"
etag "~1.8.1"
finalhandler "1.1.1"
fresh "0.5.2"
merge-descriptors "1.0.1"
methods "~1.1.2"
on-finished "~2.3.0"
parseurl "~1.3.2"
path-to-regexp "0.1.7"
proxy-addr "~2.0.3"
qs "6.5.1"
range-parser "~1.2.0"
safe-buffer "5.1.1"
send "0.16.2"
serve-static "1.13.2"
setprototypeof "1.1.0"
statuses "~1.4.0"
type-is "~1.6.16"
utils-merge "1.0.1"
vary "~1.1.2"
extend@~3.0.1:
version "3.0.2"
resolved "https://registry.yarnpkg.com/extend/-/extend-3.0.2.tgz#f8b1136b4071fbd8eb140aff858b1019ec2915fa"
extsprintf@1.3.0:
version "1.3.0"
resolved "https://registry.yarnpkg.com/extsprintf/-/extsprintf-1.3.0.tgz#96918440e3041a7a414f8c52e3c574eb3c3e1e05"
extsprintf@^1.2.0:
version "1.4.0"
resolved "https://registry.yarnpkg.com/extsprintf/-/extsprintf-1.4.0.tgz#e2689f8f356fad62cca65a3a91c5df5f9551692f"
fast-deep-equal@^1.0.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/fast-deep-equal/-/fast-deep-equal-1.1.0.tgz#c053477817c86b51daa853c81e059b733d023614"
fast-deep-equal@^2.0.1:
version "2.0.1"
resolved "https://registry.yarnpkg.com/fast-deep-equal/-/fast-deep-equal-2.0.1.tgz#7b05218ddf9667bf7f370bf7fdb2cb15fdd0aa49"
fast-json-stable-stringify@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/fast-json-stable-stringify/-/fast-json-stable-stringify-2.0.0.tgz#d5142c0caee6b1189f87d3a76111064f86c8bbf2"
fast-url-parser@1.1.3:
version "1.1.3"
resolved "https://registry.yarnpkg.com/fast-url-parser/-/fast-url-parser-1.1.3.tgz#f4af3ea9f34d8a271cf58ad2b3759f431f0b318d"
dependencies:
punycode "^1.3.2"
finalhandler@1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/finalhandler/-/finalhandler-1.1.1.tgz#eebf4ed840079c83f4249038c9d703008301b105"
dependencies:
debug "2.6.9"
encodeurl "~1.0.2"
escape-html "~1.0.3"
on-finished "~2.3.0"
parseurl "~1.3.2"
statuses "~1.4.0"
unpipe "~1.0.0"
forever-agent@~0.6.1:
version "0.6.1"
resolved "https://registry.yarnpkg.com/forever-agent/-/forever-agent-0.6.1.tgz#fbc71f0c41adeb37f96c577ad1ed42d8fdacca91"
form-data@~2.3.1:
version "2.3.2"
resolved "https://registry.yarnpkg.com/form-data/-/form-data-2.3.2.tgz#4970498be604c20c005d4f5c23aecd21d6b49099"
dependencies:
asynckit "^0.4.0"
combined-stream "1.0.6"
mime-types "^2.1.12"
forwarded@~0.1.2:
version "0.1.2"
resolved "https://registry.yarnpkg.com/forwarded/-/forwarded-0.1.2.tgz#98c23dab1175657b8c0573e8ceccd91b0ff18c84"
fresh@0.5.2:
version "0.5.2"
resolved "https://registry.yarnpkg.com/fresh/-/fresh-0.5.2.tgz#3d8cadd90d976569fa835ab1f8e4b23a105605a7"
fs-minipass@^1.2.5:
version "1.2.5"
resolved "https://registry.yarnpkg.com/fs-minipass/-/fs-minipass-1.2.5.tgz#06c277218454ec288df77ada54a03b8702aacb9d"
dependencies:
minipass "^2.2.1"
fs.realpath@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/fs.realpath/-/fs.realpath-1.0.0.tgz#1504ad2523158caa40db4a2787cb01411994ea4f"
gauge@~2.7.3:
version "2.7.4"
resolved "https://registry.yarnpkg.com/gauge/-/gauge-2.7.4.tgz#2c03405c7538c39d7eb37b317022e325fb018bf7"
dependencies:
aproba "^1.0.3"
console-control-strings "^1.0.0"
has-unicode "^2.0.0"
object-assign "^4.1.0"
signal-exit "^3.0.0"
string-width "^1.0.1"
strip-ansi "^3.0.1"
wide-align "^1.1.0"
get-func-name@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/get-func-name/-/get-func-name-2.0.0.tgz#ead774abee72e20409433a066366023dd6887a41"
get-stream@^3.0.0:
version "3.0.0"
resolved "https://registry.yarnpkg.com/get-stream/-/get-stream-3.0.0.tgz#8e943d1358dc37555054ecbe2edb05aa174ede14"
getpass@^0.1.1:
version "0.1.7"
resolved "https://registry.yarnpkg.com/getpass/-/getpass-0.1.7.tgz#5eff8e3e684d569ae4cb2b1282604e8ba62149fa"
dependencies:
assert-plus "^1.0.0"
glob-parent@^3.0.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/glob-parent/-/glob-parent-3.1.0.tgz#9e6af6299d8d3bd2bd40430832bd113df906c5ae"
dependencies:
is-glob "^3.1.0"
path-dirname "^1.0.0"
glob-slash@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/glob-slash/-/glob-slash-1.0.0.tgz#fe52efa433233f74a2fe64c7abb9bc848202ab95"
glob-slasher@1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/glob-slasher/-/glob-slasher-1.0.1.tgz#747a0e5bb222642ee10d3e05443e109493cb0f8e"
dependencies:
glob-slash "^1.0.0"
lodash.isobject "^2.4.1"
toxic "^1.0.0"
glob@7.1.2, glob@^7.0.3, glob@^7.0.5, glob@^7.1.1:
version "7.1.2"
resolved "https://registry.yarnpkg.com/glob/-/glob-7.1.2.tgz#c19c9df9a028702d678612384a6552404c636d15"
dependencies:
fs.realpath "^1.0.0"
inflight "^1.0.4"
inherits "2"
minimatch "^3.0.4"
once "^1.3.0"
path-is-absolute "^1.0.0"
globby@^6.0.0:
version "6.1.0"
resolved "https://registry.yarnpkg.com/globby/-/globby-6.1.0.tgz#f5a6d70e8395e21c858fb0489d64df02424d506c"
dependencies:
array-union "^1.0.1"
glob "^7.0.3"
object-assign "^4.0.1"
pify "^2.0.0"
pinkie-promise "^2.0.0"
growl@1.10.5:
version "1.10.5"
resolved "https://registry.yarnpkg.com/growl/-/growl-1.10.5.tgz#f2735dc2283674fa67478b10181059355c369e5e"
har-schema@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/har-schema/-/har-schema-2.0.0.tgz#a94c2224ebcac04782a0d9035521f24735b7ec92"
har-validator@~5.0.3:
version "5.0.3"
resolved "https://registry.yarnpkg.com/har-validator/-/har-validator-5.0.3.tgz#ba402c266194f15956ef15e0fcf242993f6a7dfd"
dependencies:
ajv "^5.1.0"
har-schema "^2.0.0"
has-ansi@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/has-ansi/-/has-ansi-2.0.0.tgz#34f5049ce1ecdf2b0649af3ef24e45ed35416d91"
dependencies:
ansi-regex "^2.0.0"
has-flag@^3.0.0:
version "3.0.0"
resolved "https://registry.yarnpkg.com/has-flag/-/has-flag-3.0.0.tgz#b5d454dc2199ae225699f3467e5a07f3b955bafd"
has-unicode@^2.0.0:
version "2.0.1"
resolved "https://registry.yarnpkg.com/has-unicode/-/has-unicode-2.0.1.tgz#e0e6fe6a28cf51138855e086d1691e771de2a8b9"
he@1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/he/-/he-1.1.1.tgz#93410fd21b009735151f8868c2f271f3427e23fd"
http-errors@1.6.2:
version "1.6.2"
resolved "https://registry.yarnpkg.com/http-errors/-/http-errors-1.6.2.tgz#0a002cc85707192a7e7946ceedc11155f60ec736"
dependencies:
depd "1.1.1"
inherits "2.0.3"
setprototypeof "1.0.3"
statuses ">= 1.3.1 < 2"
http-errors@~1.6.2:
version "1.6.3"
resolved "https://registry.yarnpkg.com/http-errors/-/http-errors-1.6.3.tgz#8b55680bb4be283a0b5bf4ea2e38580be1d9320d"
dependencies:
depd "~1.1.2"
inherits "2.0.3"
setprototypeof "1.1.0"
statuses ">= 1.4.0 < 2"
http-signature@~1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/http-signature/-/http-signature-1.2.0.tgz#9aecd925114772f3d95b65a60abb8f7c18fbace1"
dependencies:
assert-plus "^1.0.0"
jsprim "^1.2.2"
sshpk "^1.7.0"
iconv-lite@0.4.19:
version "0.4.19"
resolved "https://registry.yarnpkg.com/iconv-lite/-/iconv-lite-0.4.19.tgz#f7468f60135f5e5dad3399c0a81be9a1603a082b"
iconv-lite@^0.4.4:
version "0.4.23"
resolved "https://registry.yarnpkg.com/iconv-lite/-/iconv-lite-0.4.23.tgz#297871f63be507adcfbfca715d0cd0eed84e9a63"
dependencies:
safer-buffer ">= 2.1.2 < 3"
ignore-walk@^3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/ignore-walk/-/ignore-walk-3.0.1.tgz#a83e62e7d272ac0e3b551aaa82831a19b69f82f8"
dependencies:
minimatch "^3.0.4"
inflight@^1.0.4:
version "1.0.6"
resolved "https://registry.yarnpkg.com/inflight/-/inflight-1.0.6.tgz#49bd6331d7d02d0c09bc910a1075ba8165b56df9"
dependencies:
once "^1.3.0"
wrappy "1"
inherits@2, inherits@2.0.3, inherits@~2.0.3:
version "2.0.3"
resolved "https://registry.yarnpkg.com/inherits/-/inherits-2.0.3.tgz#633c2c83e3da42a502f52466022480f4208261de"
ini@~1.3.0:
version "1.3.5"
resolved "https://registry.yarnpkg.com/ini/-/ini-1.3.5.tgz#eee25f56db1c9ec6085e0c22778083f596abf927"
ipaddr.js@1.6.0:
version "1.6.0"
resolved "https://registry.yarnpkg.com/ipaddr.js/-/ipaddr.js-1.6.0.tgz#e3fa357b773da619f26e95f049d055c72796f86b"
is-extglob@^2.1.0:
version "2.1.1"
resolved "https://registry.yarnpkg.com/is-extglob/-/is-extglob-2.1.1.tgz#a88c02535791f02ed37c76a1b9ea9773c833f8c2"
is-fullwidth-code-point@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/is-fullwidth-code-point/-/is-fullwidth-code-point-1.0.0.tgz#ef9e31386f031a7f0d643af82fde50c457ef00cb"
dependencies:
number-is-nan "^1.0.0"
is-fullwidth-code-point@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/is-fullwidth-code-point/-/is-fullwidth-code-point-2.0.0.tgz#a3b30a5c4f199183167aaab93beefae3ddfb654f"
is-glob@^3.1.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/is-glob/-/is-glob-3.1.0.tgz#7ba5ae24217804ac70707b96922567486cc3e84a"
dependencies:
is-extglob "^2.1.0"
is-stream@^1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/is-stream/-/is-stream-1.1.0.tgz#12d4a3dd4e68e0b79ceb8dbc84173ae80d91ca44"
is-typedarray@~1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/is-typedarray/-/is-typedarray-1.0.0.tgz#e479c80858df0c1b11ddda6940f96011fcda4a9a"
isarray@~1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/isarray/-/isarray-1.0.0.tgz#bb935d48582cba168c06834957a54a3e07124f11"
isexe@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/isexe/-/isexe-2.0.0.tgz#e8fbf374dc556ff8947a10dcb0572d633f2cfa10"
isstream@~0.1.2:
version "0.1.2"
resolved "https://registry.yarnpkg.com/isstream/-/isstream-0.1.2.tgz#47e63f7af55afa6f92e1500e690eb8b8529c099a"
js-tokens@^3.0.2:
version "3.0.2"
resolved "https://registry.yarnpkg.com/js-tokens/-/js-tokens-3.0.2.tgz#9866df395102130e38f7f996bceb65443209c25b"
js-yaml@^3.7.0:
version "3.12.0"
resolved "https://registry.yarnpkg.com/js-yaml/-/js-yaml-3.12.0.tgz#eaed656ec8344f10f527c6bfa1b6e2244de167d1"
dependencies:
argparse "^1.0.7"
esprima "^4.0.0"
jsbn@~0.1.0:
version "0.1.1"
resolved "https://registry.yarnpkg.com/jsbn/-/jsbn-0.1.1.tgz#a5e654c2e5a2deb5f201d96cefbca80c0ef2f513"
json-schema-traverse@^0.3.0:
version "0.3.1"
resolved "https://registry.yarnpkg.com/json-schema-traverse/-/json-schema-traverse-0.3.1.tgz#349a6d44c53a51de89b40805c5d5e59b417d3340"
json-schema-traverse@^0.4.1:
version "0.4.1"
resolved "https://registry.yarnpkg.com/json-schema-traverse/-/json-schema-traverse-0.4.1.tgz#69f6a87d9513ab8bb8fe63bdb0979c448e684660"
json-schema@0.2.3:
version "0.2.3"
resolved "https://registry.yarnpkg.com/json-schema/-/json-schema-0.2.3.tgz#b480c892e59a2f05954ce727bd3f2a4e882f9e13"
json-stringify-safe@~5.0.1:
version "5.0.1"
resolved "https://registry.yarnpkg.com/json-stringify-safe/-/json-stringify-safe-5.0.1.tgz#1296a2d58fd45f19a0f6ce01d65701e2c735b6eb"
jsprim@^1.2.2:
version "1.4.1"
resolved "https://registry.yarnpkg.com/jsprim/-/jsprim-1.4.1.tgz#313e66bc1e5cc06e438bc1b7499c2e5c56acb6a2"
dependencies:
assert-plus "1.0.0"
extsprintf "1.3.0"
json-schema "0.2.3"
verror "1.10.0"
lodash._objecttypes@~2.4.1:
version "2.4.1"
resolved "https://registry.yarnpkg.com/lodash._objecttypes/-/lodash._objecttypes-2.4.1.tgz#7c0b7f69d98a1f76529f890b0cdb1b4dfec11c11"
lodash.isobject@^2.4.1:
version "2.4.1"
resolved "https://registry.yarnpkg.com/lodash.isobject/-/lodash.isobject-2.4.1.tgz#5a2e47fe69953f1ee631a7eba1fe64d2d06558f5"
dependencies:
lodash._objecttypes "~2.4.1"
lodash@^4.17.10:
version "4.17.10"
resolved "https://registry.yarnpkg.com/lodash/-/lodash-4.17.10.tgz#1b7793cf7259ea38fb3661d4d38b3260af8ae4e7"
lru-cache@^4.0.1:
version "4.1.3"
resolved "https://registry.yarnpkg.com/lru-cache/-/lru-cache-4.1.3.tgz#a1175cf3496dfc8436c156c334b4955992bce69c"
dependencies:
pseudomap "^1.0.2"
yallist "^2.1.2"
make-error@^1.1.1:
version "1.3.4"
resolved "https://registry.yarnpkg.com/make-error/-/make-error-1.3.4.tgz#19978ed575f9e9545d2ff8c13e33b5d18a67d535"
media-typer@0.3.0:
version "0.3.0"
resolved "https://registry.yarnpkg.com/media-typer/-/media-typer-0.3.0.tgz#8710d7af0aa626f8fffa1ce00168545263255748"
merge-descriptors@1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/merge-descriptors/-/merge-descriptors-1.0.1.tgz#b00aaa556dd8b44568150ec9d1b953f3f90cbb61"
methods@~1.1.2:
version "1.1.2"
resolved "https://registry.yarnpkg.com/methods/-/methods-1.1.2.tgz#5529a4d67654134edcc5266656835b0f851afcee"
mime-db@~1.33.0:
version "1.33.0"
resolved "https://registry.yarnpkg.com/mime-db/-/mime-db-1.33.0.tgz#a3492050a5cb9b63450541e39d9788d2272783db"
mime-db@~1.35.0:
version "1.35.0"
resolved "https://registry.yarnpkg.com/mime-db/-/mime-db-1.35.0.tgz#0569d657466491283709663ad379a99b90d9ab47"
mime-types@2.1.18:
version "2.1.18"
resolved "https://registry.yarnpkg.com/mime-types/-/mime-types-2.1.18.tgz#6f323f60a83d11146f831ff11fd66e2fe5503bb8"
dependencies:
mime-db "~1.33.0"
mime-types@^2.1.12, mime-types@~2.1.17, mime-types@~2.1.18:
version "2.1.19"
resolved "https://registry.yarnpkg.com/mime-types/-/mime-types-2.1.19.tgz#71e464537a7ef81c15f2db9d97e913fc0ff606f0"
dependencies:
mime-db "~1.35.0"
mime@1.4.1:
version "1.4.1"
resolved "https://registry.yarnpkg.com/mime/-/mime-1.4.1.tgz#121f9ebc49e3766f311a76e1fa1c8003c4b03aa6"
minimatch@3.0.4, minimatch@^3.0.4:
version "3.0.4"
resolved "https://registry.yarnpkg.com/minimatch/-/minimatch-3.0.4.tgz#5166e286457f03306064be5497e8dbb0c3d32083"
dependencies:
brace-expansion "^1.1.7"
minimist@0.0.8:
version "0.0.8"
resolved "https://registry.yarnpkg.com/minimist/-/minimist-0.0.8.tgz#857fcabfc3397d2625b8228262e86aa7a011b05d"
minimist@^1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/minimist/-/minimist-1.2.0.tgz#a35008b20f41383eec1fb914f4cd5df79a264284"
minipass@^2.2.1, minipass@^2.3.3:
version "2.3.3"
resolved "https://registry.yarnpkg.com/minipass/-/minipass-2.3.3.tgz#a7dcc8b7b833f5d368759cce544dccb55f50f233"
dependencies:
safe-buffer "^5.1.2"
yallist "^3.0.0"
minizlib@^1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/minizlib/-/minizlib-1.1.0.tgz#11e13658ce46bc3a70a267aac58359d1e0c29ceb"
dependencies:
minipass "^2.2.1"
mkdirp@0.5.1, mkdirp@^0.5.0, mkdirp@^0.5.1:
version "0.5.1"
resolved "https://registry.yarnpkg.com/mkdirp/-/mkdirp-0.5.1.tgz#30057438eac6cf7f8c4767f38648d6697d75c903"
dependencies:
minimist "0.0.8"
mocha@^5.2.0:
version "5.2.0"
resolved "https://registry.yarnpkg.com/mocha/-/mocha-5.2.0.tgz#6d8ae508f59167f940f2b5b3c4a612ae50c90ae6"
dependencies:
browser-stdout "1.3.1"
commander "2.15.1"
debug "3.1.0"
diff "3.5.0"
escape-string-regexp "1.0.5"
glob "7.1.2"
growl "1.10.5"
he "1.1.1"
minimatch "3.0.4"
mkdirp "0.5.1"
supports-color "5.4.0"
ms@2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/ms/-/ms-2.0.0.tgz#5608aeadfc00be6c2901df5f9861788de0d597c8"
nan@~2.10.0:
version "2.10.0"
resolved "https://registry.yarnpkg.com/nan/-/nan-2.10.0.tgz#96d0cd610ebd58d4b4de9cc0c6828cda99c7548f"
needle@^2.2.1:
version "2.2.1"
resolved "https://registry.yarnpkg.com/needle/-/needle-2.2.1.tgz#b5e325bd3aae8c2678902fa296f729455d1d3a7d"
dependencies:
debug "^2.1.2"
iconv-lite "^0.4.4"
sax "^1.2.4"
negotiator@0.6.1:
version "0.6.1"
resolved "https://registry.yarnpkg.com/negotiator/-/negotiator-0.6.1.tgz#2b327184e8992101177b28563fb5e7102acd0ca9"
node-nvidia-smi@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/node-nvidia-smi/-/node-nvidia-smi-1.0.0.tgz#6aa57574540b2bed91c9a80218516ffa686e5ac7"
dependencies:
xml2js "^0.4.17"
node-pre-gyp@^0.10.3:
version "0.10.3"
resolved "https://registry.yarnpkg.com/node-pre-gyp/-/node-pre-gyp-0.10.3.tgz#3070040716afdc778747b61b6887bf78880b80fc"
dependencies:
detect-libc "^1.0.2"
mkdirp "^0.5.1"
needle "^2.2.1"
nopt "^4.0.1"
npm-packlist "^1.1.6"
npmlog "^4.0.2"
rc "^1.2.7"
rimraf "^2.6.1"
semver "^5.3.0"
tar "^4"
node-version@^1.0.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/node-version/-/node-version-1.2.0.tgz#34fde3ffa8e1149bd323983479dda620e1b5060d"
nopt@^4.0.1:
version "4.0.1"
resolved "https://registry.yarnpkg.com/nopt/-/nopt-4.0.1.tgz#d0d4685afd5415193c8c7505602d0d17cd64474d"
dependencies:
abbrev "1"
osenv "^0.1.4"
npm-bundled@^1.0.1:
version "1.0.3"
resolved "https://registry.yarnpkg.com/npm-bundled/-/npm-bundled-1.0.3.tgz#7e71703d973af3370a9591bafe3a63aca0be2308"
npm-packlist@^1.1.6:
version "1.1.11"
resolved "https://registry.yarnpkg.com/npm-packlist/-/npm-packlist-1.1.11.tgz#84e8c683cbe7867d34b1d357d893ce29e28a02de"
dependencies:
ignore-walk "^3.0.1"
npm-bundled "^1.0.1"
npm-run-path@^2.0.0:
version "2.0.2"
resolved "https://registry.yarnpkg.com/npm-run-path/-/npm-run-path-2.0.2.tgz#35a9232dfa35d7067b4cb2ddf2357b1871536c5f"
dependencies:
path-key "^2.0.0"
npmlog@^4.0.2:
version "4.1.2"
resolved "https://registry.yarnpkg.com/npmlog/-/npmlog-4.1.2.tgz#08a7f2a8bf734604779a9efa4ad5cc717abb954b"
dependencies:
are-we-there-yet "~1.1.2"
console-control-strings "~1.1.0"
gauge "~2.7.3"
set-blocking "~2.0.0"
number-is-nan@^1.0.0:
version "1.0.1"
resolved "https://registry.yarnpkg.com/number-is-nan/-/number-is-nan-1.0.1.tgz#097b602b53422a522c1afb8790318336941a011d"
oauth-sign@~0.8.2:
version "0.8.2"
resolved "https://registry.yarnpkg.com/oauth-sign/-/oauth-sign-0.8.2.tgz#46a6ab7f0aead8deae9ec0565780b7d4efeb9d43"
object-assign@^4.0.1, object-assign@^4.1.0:
version "4.1.1"
resolved "https://registry.yarnpkg.com/object-assign/-/object-assign-4.1.1.tgz#2109adc7965887cfc05cbbd442cac8bfbb360863"
on-finished@~2.3.0:
version "2.3.0"
resolved "https://registry.yarnpkg.com/on-finished/-/on-finished-2.3.0.tgz#20f1336481b083cd75337992a16971aa2d906947"
dependencies:
ee-first "1.1.1"
once@^1.3.0:
version "1.4.0"
resolved "https://registry.yarnpkg.com/once/-/once-1.4.0.tgz#583b1aa775961d4b113ac17d9c50baef9dd76bd1"
dependencies:
wrappy "1"
os-homedir@^1.0.0:
version "1.0.2"
resolved "https://registry.yarnpkg.com/os-homedir/-/os-homedir-1.0.2.tgz#ffbc4988336e0e833de0c168c7ef152121aa7fb3"
os-tmpdir@^1.0.0, os-tmpdir@~1.0.2:
version "1.0.2"
resolved "https://registry.yarnpkg.com/os-tmpdir/-/os-tmpdir-1.0.2.tgz#bbe67406c79aa85c5cfec766fe5734555dfa1274"
osenv@^0.1.4:
version "0.1.5"
resolved "https://registry.yarnpkg.com/osenv/-/osenv-0.1.5.tgz#85cdfafaeb28e8677f416e287592b5f3f49ea410"
dependencies:
os-homedir "^1.0.0"
os-tmpdir "^1.0.0"
p-finally@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/p-finally/-/p-finally-1.0.0.tgz#3fbcfb15b899a44123b34b6dcc18b724336a2cae"
parent-module@^0.1.0:
version "0.1.0"
resolved "https://registry.yarnpkg.com/parent-module/-/parent-module-0.1.0.tgz#b5292863a1e8c476ecf857e7d75c98920b24b8a6"
dependencies:
callsites "^1.0.0"
parseurl@~1.3.2:
version "1.3.2"
resolved "https://registry.yarnpkg.com/parseurl/-/parseurl-1.3.2.tgz#fc289d4ed8993119460c156253262cdc8de65bf3"
path-dirname@^1.0.0:
version "1.0.2"
resolved "https://registry.yarnpkg.com/path-dirname/-/path-dirname-1.0.2.tgz#cc33d24d525e099a5388c0336c6e32b9160609e0"
path-is-absolute@^1.0.0:
version "1.0.1"
resolved "https://registry.yarnpkg.com/path-is-absolute/-/path-is-absolute-1.0.1.tgz#174b9268735534ffbc7ace6bf53a5a9e1b5c5f5f"
path-is-inside@1.0.2:
version "1.0.2"
resolved "https://registry.yarnpkg.com/path-is-inside/-/path-is-inside-1.0.2.tgz#365417dede44430d1c11af61027facf074bdfc53"
path-key@^2.0.0:
version "2.0.1"
resolved "https://registry.yarnpkg.com/path-key/-/path-key-2.0.1.tgz#411cadb574c5a140d3a4b1910d40d80cc9f40b40"
path-parse@^1.0.5:
version "1.0.5"
resolved "https://registry.yarnpkg.com/path-parse/-/path-parse-1.0.5.tgz#3c1adf871ea9cd6c9431b6ea2bd74a0ff055c4c1"
path-to-regexp@0.1.7:
version "0.1.7"
resolved "https://registry.yarnpkg.com/path-to-regexp/-/path-to-regexp-0.1.7.tgz#df604178005f522f15eb4490e7247a1bfaa67f8c"
path-to-regexp@2.2.1:
version "2.2.1"
resolved "https://registry.yarnpkg.com/path-to-regexp/-/path-to-regexp-2.2.1.tgz#90b617025a16381a879bc82a38d4e8bdeb2bcf45"
pathval@^1.0.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/pathval/-/pathval-1.1.0.tgz#b942e6d4bde653005ef6b71361def8727d0645e0"
performance-now@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/performance-now/-/performance-now-2.1.0.tgz#6309f4e0e5fa913ec1c69307ae364b4b377c9e7b"
pify@^2.0.0:
version "2.3.0"
resolved "https://registry.yarnpkg.com/pify/-/pify-2.3.0.tgz#ed141a6ac043a849ea588498e7dca8b15330e90c"
pinkie-promise@^2.0.0:
version "2.0.1"
resolved "https://registry.yarnpkg.com/pinkie-promise/-/pinkie-promise-2.0.1.tgz#2135d6dfa7a358c069ac9b178776288228450ffa"
dependencies:
pinkie "^2.0.0"
pinkie@^2.0.0:
version "2.0.4"
resolved "https://registry.yarnpkg.com/pinkie/-/pinkie-2.0.4.tgz#72556b80cfa0d48a974e80e77248e80ed4f7f870"
process-nextick-args@~2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/process-nextick-args/-/process-nextick-args-2.0.0.tgz#a37d732f4271b4ab1ad070d35508e8290788ffaa"
promise-polyfill@^6.0.1:
version "6.1.0"
resolved "https://registry.yarnpkg.com/promise-polyfill/-/promise-polyfill-6.1.0.tgz#dfa96943ea9c121fca4de9b5868cb39d3472e057"
proxy-addr@~2.0.3:
version "2.0.3"
resolved "https://registry.yarnpkg.com/proxy-addr/-/proxy-addr-2.0.3.tgz#355f262505a621646b3130a728eb647e22055341"
dependencies:
forwarded "~0.1.2"
ipaddr.js "1.6.0"
pseudomap@^1.0.2:
version "1.0.2"
resolved "https://registry.yarnpkg.com/pseudomap/-/pseudomap-1.0.2.tgz#f052a28da70e618917ef0a8ac34c1ae5a68286b3"
punycode@^1.3.2, punycode@^1.4.1:
version "1.4.1"
resolved "https://registry.yarnpkg.com/punycode/-/punycode-1.4.1.tgz#c0d5a63b2718800ad8e1eb0fa5269c84dd41845e"
punycode@^2.1.0:
version "2.1.1"
resolved "https://registry.yarnpkg.com/punycode/-/punycode-2.1.1.tgz#b58b010ac40c22c5657616c8d2c2c02c7bf479ec"
qs@6.5.1:
version "6.5.1"
resolved "https://registry.yarnpkg.com/qs/-/qs-6.5.1.tgz#349cdf6eef89ec45c12d7d5eb3fc0c870343a6d8"
qs@~6.5.1:
version "6.5.2"
resolved "https://registry.yarnpkg.com/qs/-/qs-6.5.2.tgz#cb3ae806e8740444584ef154ce8ee98d403f3e36"
range-parser@1.2.0, range-parser@~1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/range-parser/-/range-parser-1.2.0.tgz#f49be6b487894ddc40dcc94a322f611092e00d5e"
raw-body@2.3.2:
version "2.3.2"
resolved "https://registry.yarnpkg.com/raw-body/-/raw-body-2.3.2.tgz#bcd60c77d3eb93cde0050295c3f379389bc88f89"
dependencies:
bytes "3.0.0"
http-errors "1.6.2"
iconv-lite "0.4.19"
unpipe "1.0.0"
rc@^1.0.1, rc@^1.1.6, rc@^1.2.7:
version "1.2.8"
resolved "https://registry.yarnpkg.com/rc/-/rc-1.2.8.tgz#cd924bf5200a075b83c188cd6b9e211b7fc0d3ed"
dependencies:
deep-extend "^0.6.0"
ini "~1.3.0"
minimist "^1.2.0"
strip-json-comments "~2.0.1"
readable-stream@^2.0.6:
version "2.3.6"
resolved "https://registry.yarnpkg.com/readable-stream/-/readable-stream-2.3.6.tgz#b11c27d88b8ff1fbe070643cf94b0c79ae1b0aaf"
dependencies:
core-util-is "~1.0.0"
inherits "~2.0.3"
isarray "~1.0.0"
process-nextick-args "~2.0.0"
safe-buffer "~5.1.1"
string_decoder "~1.1.1"
util-deprecate "~1.0.1"
reflect-metadata@^0.1.10:
version "0.1.12"
resolved "https://registry.yarnpkg.com/reflect-metadata/-/reflect-metadata-0.1.12.tgz#311bf0c6b63cd782f228a81abe146a2bfa9c56f2"
registry-auth-token@3.3.2:
version "3.3.2"
resolved "https://registry.yarnpkg.com/registry-auth-token/-/registry-auth-token-3.3.2.tgz#851fd49038eecb586911115af845260eec983f20"
dependencies:
rc "^1.1.6"
safe-buffer "^5.0.1"
registry-url@3.1.0:
version "3.1.0"
resolved "https://registry.yarnpkg.com/registry-url/-/registry-url-3.1.0.tgz#3d4ef870f73dde1d77f0cf9a381432444e174942"
dependencies:
rc "^1.0.1"
request@^2.87.0:
version "2.87.0"
resolved "https://registry.yarnpkg.com/request/-/request-2.87.0.tgz#32f00235cd08d482b4d0d68db93a829c0ed5756e"
dependencies:
aws-sign2 "~0.7.0"
aws4 "^1.6.0"
caseless "~0.12.0"
combined-stream "~1.0.5"
extend "~3.0.1"
forever-agent "~0.6.1"
form-data "~2.3.1"
har-validator "~5.0.3"
http-signature "~1.2.0"
is-typedarray "~1.0.0"
isstream "~0.1.2"
json-stringify-safe "~5.0.1"
mime-types "~2.1.17"
oauth-sign "~0.8.2"
performance-now "^2.1.0"
qs "~6.5.1"
safe-buffer "^5.1.1"
tough-cookie "~2.3.3"
tunnel-agent "^0.6.0"
uuid "^3.1.0"
require-glob@^3.2.0:
version "3.2.0"
resolved "https://registry.yarnpkg.com/require-glob/-/require-glob-3.2.0.tgz#90bfe2c8efb4b9f972eb9a3f5e580832e04f64d3"
dependencies:
glob-parent "^3.0.0"
globby "^6.0.0"
parent-module "^0.1.0"
resolve@^1.3.2:
version "1.8.1"
resolved "https://registry.yarnpkg.com/resolve/-/resolve-1.8.1.tgz#82f1ec19a423ac1fbd080b0bab06ba36e84a7a26"
dependencies:
path-parse "^1.0.5"
rimraf@^2.6.1:
version "2.6.2"
resolved "https://registry.yarnpkg.com/rimraf/-/rimraf-2.6.2.tgz#2ed8150d24a16ea8651e6d6ef0f47c4158ce7a36"
dependencies:
glob "^7.0.5"
rx@^4.1.0:
version "4.1.0"
resolved "https://registry.yarnpkg.com/rx/-/rx-4.1.0.tgz#a5f13ff79ef3b740fe30aa803fb09f98805d4782"
safe-buffer@5.1.1:
version "5.1.1"
resolved "https://registry.yarnpkg.com/safe-buffer/-/safe-buffer-5.1.1.tgz#893312af69b2123def71f57889001671eeb2c853"
safe-buffer@^5.0.1, safe-buffer@^5.1.1, safe-buffer@^5.1.2, safe-buffer@~5.1.0, safe-buffer@~5.1.1:
version "5.1.2"
resolved "https://registry.yarnpkg.com/safe-buffer/-/safe-buffer-5.1.2.tgz#991ec69d296e0313747d59bdfd2b745c35f8828d"
"safer-buffer@>= 2.1.2 < 3", safer-buffer@^2.0.2:
version "2.1.2"
resolved "https://registry.yarnpkg.com/safer-buffer/-/safer-buffer-2.1.2.tgz#44fa161b0187b9549dd84bb91802f9bd8385cd6a"
sax@>=0.6.0, sax@^1.2.4:
version "1.2.4"
resolved "https://registry.yarnpkg.com/sax/-/sax-1.2.4.tgz#2816234e2378bddc4e5354fab5caa895df7100d9"
semver@^5.1.0, semver@^5.3.0:
version "5.5.0"
resolved "https://registry.yarnpkg.com/semver/-/semver-5.5.0.tgz#dc4bbc7a6ca9d916dee5d43516f0092b58f7b8ab"
send@0.16.2:
version "0.16.2"
resolved "https://registry.yarnpkg.com/send/-/send-0.16.2.tgz#6ecca1e0f8c156d141597559848df64730a6bbc1"
dependencies:
debug "2.6.9"
depd "~1.1.2"
destroy "~1.0.4"
encodeurl "~1.0.2"
escape-html "~1.0.3"
etag "~1.8.1"
fresh "0.5.2"
http-errors "~1.6.2"
mime "1.4.1"
ms "2.0.0"
on-finished "~2.3.0"
range-parser "~1.2.0"
statuses "~1.4.0"
serve-handler@4.0.0:
version "4.0.0"
resolved "https://registry.yarnpkg.com/serve-handler/-/serve-handler-4.0.0.tgz#cf2a40f27f7a5ec29cbbd1732fcd6a16afdc0c41"
dependencies:
bytes "3.0.0"
content-disposition "0.5.2"
fast-url-parser "1.1.3"
glob-slasher "1.0.1"
mime-types "2.1.18"
minimatch "3.0.4"
path-is-inside "1.0.2"
path-to-regexp "2.2.1"
range-parser "1.2.0"
serve-static@1.13.2:
version "1.13.2"
resolved "https://registry.yarnpkg.com/serve-static/-/serve-static-1.13.2.tgz#095e8472fd5b46237db50ce486a43f4b86c6cec1"
dependencies:
encodeurl "~1.0.2"
escape-html "~1.0.3"
parseurl "~1.3.2"
send "0.16.2"
serve@^9.6.0:
version "9.6.0"
resolved "https://registry.yarnpkg.com/serve/-/serve-9.6.0.tgz#303f198c03ad2d47eb90972447c7bd5878a4c7ac"
dependencies:
"@zeit/schemas" "2.1.1"
ajv "6.5.2"
arg "2.0.0"
boxen "1.3.0"
chalk "2.4.1"
clipboardy "1.2.3"
serve-handler "4.0.0"
update-check "1.5.2"
set-blocking@~2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/set-blocking/-/set-blocking-2.0.0.tgz#045f9782d011ae9a6803ddd382b24392b3d890f7"
setprototypeof@1.0.3:
version "1.0.3"
resolved "https://registry.yarnpkg.com/setprototypeof/-/setprototypeof-1.0.3.tgz#66567e37043eeb4f04d91bd658c0cbefb55b8e04"
setprototypeof@1.1.0:
version "1.1.0"
resolved "https://registry.yarnpkg.com/setprototypeof/-/setprototypeof-1.1.0.tgz#d0bd85536887b6fe7c0d818cb962d9d91c54e656"
shebang-command@^1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/shebang-command/-/shebang-command-1.2.0.tgz#44aac65b695b03398968c39f363fee5deafdf1ea"
dependencies:
shebang-regex "^1.0.0"
shebang-regex@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/shebang-regex/-/shebang-regex-1.0.0.tgz#da42f49740c0b42db2ca9728571cb190c98efea3"
signal-exit@^3.0.0:
version "3.0.2"
resolved "https://registry.yarnpkg.com/signal-exit/-/signal-exit-3.0.2.tgz#b5fdc08f1287ea1178628e415e25132b73646c6d"
source-map-support@^0.5.6:
version "0.5.6"
resolved "https://registry.yarnpkg.com/source-map-support/-/source-map-support-0.5.6.tgz#4435cee46b1aab62b8e8610ce60f788091c51c13"
dependencies:
buffer-from "^1.0.0"
source-map "^0.6.0"
source-map@^0.6.0:
version "0.6.1"
resolved "https://registry.yarnpkg.com/source-map/-/source-map-0.6.1.tgz#74722af32e9614e9c287a8d0bbde48b5e2f1a263"
sprintf-js@~1.0.2:
version "1.0.3"
resolved "https://registry.yarnpkg.com/sprintf-js/-/sprintf-js-1.0.3.tgz#04e6926f662895354f3dd015203633b857297e2c"
sqlite3@^4.0.2:
version "4.0.2"
resolved "https://registry.yarnpkg.com/sqlite3/-/sqlite3-4.0.2.tgz#1bbeb68b03ead5d499e42a3a1b140064791c5a64"
dependencies:
nan "~2.10.0"
node-pre-gyp "^0.10.3"
request "^2.87.0"
ssh2-streams@~0.2.0:
version "0.2.1"
resolved "https://registry.yarnpkg.com/ssh2-streams/-/ssh2-streams-0.2.1.tgz#9c9c9964be60e9644575af328677f64b1e5cbd79"
dependencies:
asn1 "~0.2.0"
semver "^5.1.0"
streamsearch "~0.1.2"
ssh2@^0.6.1:
version "0.6.1"
resolved "https://registry.yarnpkg.com/ssh2/-/ssh2-0.6.1.tgz#5dde1a7394bb978b1f9c2f014affee2f5493bd40"
dependencies:
ssh2-streams "~0.2.0"
sshpk@^1.7.0:
version "1.14.2"
resolved "https://registry.yarnpkg.com/sshpk/-/sshpk-1.14.2.tgz#c6fc61648a3d9c4e764fd3fcdf4ea105e492ba98"
dependencies:
asn1 "~0.2.3"
assert-plus "^1.0.0"
dashdash "^1.12.0"
getpass "^0.1.1"
safer-buffer "^2.0.2"
optionalDependencies:
bcrypt-pbkdf "^1.0.0"
ecc-jsbn "~0.1.1"
jsbn "~0.1.0"
tweetnacl "~0.14.0"
"statuses@>= 1.3.1 < 2", "statuses@>= 1.4.0 < 2":
version "1.5.0"
resolved "https://registry.yarnpkg.com/statuses/-/statuses-1.5.0.tgz#161c7dac177659fd9811f43771fa99381478628c"
statuses@~1.4.0:
version "1.4.0"
resolved "https://registry.yarnpkg.com/statuses/-/statuses-1.4.0.tgz#bb73d446da2796106efcc1b601a253d6c46bd087"
stream-buffers@^3.0.2:
version "3.0.2"
resolved "https://registry.yarnpkg.com/stream-buffers/-/stream-buffers-3.0.2.tgz#5249005a8d5c2d00b3a32e6e0a6ea209dc4f3521"
streamsearch@~0.1.2:
version "0.1.2"
resolved "https://registry.yarnpkg.com/streamsearch/-/streamsearch-0.1.2.tgz#808b9d0e56fc273d809ba57338e929919a1a9f1a"
string-width@^1.0.1:
version "1.0.2"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-1.0.2.tgz#118bdf5b8cdc51a2a7e70d211e07e2b0b9b107d3"
dependencies:
code-point-at "^1.0.0"
is-fullwidth-code-point "^1.0.0"
strip-ansi "^3.0.0"
"string-width@^1.0.2 || 2", string-width@^2.0.0, string-width@^2.1.1:
version "2.1.1"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-2.1.1.tgz#ab93f27a8dc13d28cac815c462143a6d9012ae9e"
dependencies:
is-fullwidth-code-point "^2.0.0"
strip-ansi "^4.0.0"
string_decoder@~1.1.1:
version "1.1.1"
resolved "https://registry.yarnpkg.com/string_decoder/-/string_decoder-1.1.1.tgz#9cf1611ba62685d7030ae9e4ba34149c3af03fc8"
dependencies:
safe-buffer "~5.1.0"
strip-ansi@^3.0.0, strip-ansi@^3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-3.0.1.tgz#6a385fb8853d952d5ff05d0e8aaf94278dc63dcf"
dependencies:
ansi-regex "^2.0.0"
strip-ansi@^4.0.0:
version "4.0.0"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-4.0.0.tgz#a8479022eb1ac368a871389b635262c505ee368f"
dependencies:
ansi-regex "^3.0.0"
strip-eof@^1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/strip-eof/-/strip-eof-1.0.0.tgz#bb43ff5598a6eb05d89b59fcd129c983313606bf"
strip-json-comments@~2.0.1:
version "2.0.1"
resolved "https://registry.yarnpkg.com/strip-json-comments/-/strip-json-comments-2.0.1.tgz#3c531942e908c2697c0ec344858c286c7ca0a60a"
supports-color@5.4.0, supports-color@^5.3.0:
version "5.4.0"
resolved "https://registry.yarnpkg.com/supports-color/-/supports-color-5.4.0.tgz#1c6b337402c2137605efe19f10fec390f6faab54"
dependencies:
has-flag "^3.0.0"
supports-color@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/supports-color/-/supports-color-2.0.0.tgz#535d045ce6b6363fa40117084629995e9df324c7"
tail-stream@^0.3.4:
version "0.3.4"
resolved "https://registry.yarnpkg.com/tail-stream/-/tail-stream-0.3.4.tgz#bc675a20e92732b1a6a7cc65d6be66f7817fd5c1"
tar@^4:
version "4.4.4"
resolved "https://registry.yarnpkg.com/tar/-/tar-4.4.4.tgz#ec8409fae9f665a4355cc3b4087d0820232bb8cd"
dependencies:
chownr "^1.0.1"
fs-minipass "^1.2.5"
minipass "^2.3.3"
minizlib "^1.1.0"
mkdirp "^0.5.0"
safe-buffer "^5.1.2"
yallist "^3.0.2"
term-size@^1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/term-size/-/term-size-1.2.0.tgz#458b83887f288fc56d6fffbfad262e26638efa69"
dependencies:
execa "^0.7.0"
tmp@^0.0.33:
version "0.0.33"
resolved "https://registry.yarnpkg.com/tmp/-/tmp-0.0.33.tgz#6d34335889768d21b2bcda0aa277ced3b1bfadf9"
dependencies:
os-tmpdir "~1.0.2"
tough-cookie@~2.3.3:
version "2.3.4"
resolved "https://registry.yarnpkg.com/tough-cookie/-/tough-cookie-2.3.4.tgz#ec60cee38ac675063ffc97a5c18970578ee83655"
dependencies:
punycode "^1.4.1"
tree-kill@^1.2.0:
version "1.2.0"
resolved "https://registry.yarnpkg.com/tree-kill/-/tree-kill-1.2.0.tgz#5846786237b4239014f05db156b643212d4c6f36"
toxic@^1.0.0:
version "1.0.1"
resolved "https://registry.yarnpkg.com/toxic/-/toxic-1.0.1.tgz#8c2e2528da591100adc3883f2c0e56acfb1c7288"
dependencies:
lodash "^4.17.10"
ts-deferred@^1.0.4:
version "1.0.4"
resolved "https://registry.yarnpkg.com/ts-deferred/-/ts-deferred-1.0.4.tgz#58145ebaeef5b8f2a290b8cec3d060839f9489c7"
ts-node@^7.0.0:
version "7.0.0"
resolved "https://registry.yarnpkg.com/ts-node/-/ts-node-7.0.0.tgz#a94a13c75e5e1aa6b82814b84c68deb339ba7bff"
dependencies:
arrify "^1.0.0"
buffer-from "^1.1.0"
diff "^3.1.0"
make-error "^1.1.1"
minimist "^1.2.0"
mkdirp "^0.5.1"
source-map-support "^0.5.6"
yn "^2.0.0"
tslib@^1.8.0, tslib@^1.8.1:
version "1.9.3"
resolved "https://registry.yarnpkg.com/tslib/-/tslib-1.9.3.tgz#d7e4dd79245d85428c4d7e4822a79917954ca286"
tslint-microsoft-contrib@^5.1.0:
version "5.1.0"
resolved "https://registry.yarnpkg.com/tslint-microsoft-contrib/-/tslint-microsoft-contrib-5.1.0.tgz#777c32d51aba16f4565e47aac749a1631176cd9f"
dependencies:
tsutils "^2.12.1"
tslint@^5.11.0:
version "5.11.0"
resolved "https://registry.yarnpkg.com/tslint/-/tslint-5.11.0.tgz#98f30c02eae3cde7006201e4c33cb08b48581eed"
dependencies:
babel-code-frame "^6.22.0"
builtin-modules "^1.1.1"
chalk "^2.3.0"
commander "^2.12.1"
diff "^3.2.0"
glob "^7.1.1"
js-yaml "^3.7.0"
minimatch "^3.0.4"
resolve "^1.3.2"
semver "^5.3.0"
tslib "^1.8.0"
tsutils "^2.27.2"
tsutils@^2.12.1, tsutils@^2.27.2:
version "2.29.0"
resolved "https://registry.yarnpkg.com/tsutils/-/tsutils-2.29.0.tgz#32b488501467acbedd4b85498673a0812aca0b99"
dependencies:
tslib "^1.8.1"
tunnel-agent@^0.6.0:
version "0.6.0"
resolved "https://registry.yarnpkg.com/tunnel-agent/-/tunnel-agent-0.6.0.tgz#27a5dea06b36b04a0a9966774b290868f0fc40fd"
dependencies:
safe-buffer "^5.0.1"
tweetnacl@^0.14.3, tweetnacl@~0.14.0:
version "0.14.5"
resolved "https://registry.yarnpkg.com/tweetnacl/-/tweetnacl-0.14.5.tgz#5ae68177f192d4456269d108afa93ff8743f4f64"
type-detect@^4.0.0:
version "4.0.8"
resolved "https://registry.yarnpkg.com/type-detect/-/type-detect-4.0.8.tgz#7646fb5f18871cfbb7749e69bd39a6388eb7450c"
type-is@~1.6.15, type-is@~1.6.16:
version "1.6.16"
resolved "https://registry.yarnpkg.com/type-is/-/type-is-1.6.16.tgz#f89ce341541c672b25ee7ae3c73dee3b2be50194"
dependencies:
media-typer "0.3.0"
mime-types "~2.1.18"
typescript-ioc@^1.2.4:
version "1.2.4"
resolved "https://registry.yarnpkg.com/typescript-ioc/-/typescript-ioc-1.2.4.tgz#21290097b163632de58a3abba7553daef8651f49"
dependencies:
reflect-metadata "^0.1.10"
require-glob "^3.2.0"
typescript-string-operations@^1.3.1:
version "1.3.1"
resolved "https://registry.yarnpkg.com/typescript-string-operations/-/typescript-string-operations-1.3.1.tgz#461b886cc9ccd4dd16810b1f248b2e6f6580956b"
typescript@^3.0.1:
version "3.0.1"
resolved "https://registry.yarnpkg.com/typescript/-/typescript-3.0.1.tgz#43738f29585d3a87575520a4b93ab6026ef11fdb"
unpipe@1.0.0, unpipe@~1.0.0:
version "1.0.0"
resolved "https://registry.yarnpkg.com/unpipe/-/unpipe-1.0.0.tgz#b2bf4ee8514aae6165b4817829d21b2ef49904ec"
update-check@1.5.2:
version "1.5.2"
resolved "https://registry.yarnpkg.com/update-check/-/update-check-1.5.2.tgz#2fe09f725c543440b3d7dabe8971f2d5caaedc28"
dependencies:
registry-auth-token "3.3.2"
registry-url "3.1.0"
uri-js@^4.2.1:
version "4.2.2"
resolved "https://registry.yarnpkg.com/uri-js/-/uri-js-4.2.2.tgz#94c540e1ff772956e2299507c010aea6c8838eb0"
dependencies:
punycode "^2.1.0"
util-deprecate@~1.0.1:
version "1.0.2"
resolved "https://registry.yarnpkg.com/util-deprecate/-/util-deprecate-1.0.2.tgz#450d4dc9fa70de732762fbd2d4a28981419a0ccf"
utils-merge@1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/utils-merge/-/utils-merge-1.0.1.tgz#9f95710f50a267947b2ccc124741c1028427e713"
uuid@^3.1.0:
version "3.3.2"
resolved "https://registry.yarnpkg.com/uuid/-/uuid-3.3.2.tgz#1b4af4955eb3077c501c23872fc6513811587131"
vary@~1.1.2:
version "1.1.2"
resolved "https://registry.yarnpkg.com/vary/-/vary-1.1.2.tgz#2299f02c6ded30d4a5961b0b9f74524a18f634fc"
verror@1.10.0:
version "1.10.0"
resolved "https://registry.yarnpkg.com/verror/-/verror-1.10.0.tgz#3a105ca17053af55d6e270c1f8288682e18da400"
dependencies:
assert-plus "^1.0.0"
core-util-is "1.0.2"
extsprintf "^1.2.0"
which@^1.2.9:
version "1.3.1"
resolved "https://registry.yarnpkg.com/which/-/which-1.3.1.tgz#a45043d54f5805316da8d62f9f50918d3da70b0a"
dependencies:
isexe "^2.0.0"
wide-align@^1.1.0:
version "1.1.3"
resolved "https://registry.yarnpkg.com/wide-align/-/wide-align-1.1.3.tgz#ae074e6bdc0c14a431e804e624549c633b000457"
dependencies:
string-width "^1.0.2 || 2"
widest-line@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/widest-line/-/widest-line-2.0.0.tgz#0142a4e8a243f8882c0233aa0e0281aa76152273"
dependencies:
string-width "^2.1.1"
wrappy@1:
version "1.0.2"
resolved "https://registry.yarnpkg.com/wrappy/-/wrappy-1.0.2.tgz#b5243d8f3ec1aa35f1364605bc0d1036e30ab69f"
xml2js@^0.4.17:
version "0.4.19"
resolved "https://registry.yarnpkg.com/xml2js/-/xml2js-0.4.19.tgz#686c20f213209e94abf0d1bcf1efaa291c7827a7"
dependencies:
sax ">=0.6.0"
xmlbuilder "~9.0.1"
xmlbuilder@~9.0.1:
version "9.0.7"
resolved "https://registry.yarnpkg.com/xmlbuilder/-/xmlbuilder-9.0.7.tgz#132ee63d2ec5565c557e20f4c22df9aca686b10d"
yallist@^2.1.2:
version "2.1.2"
resolved "https://registry.yarnpkg.com/yallist/-/yallist-2.1.2.tgz#1c11f9218f076089a47dd512f93c6699a6a81d52"
yallist@^3.0.0, yallist@^3.0.2:
version "3.0.2"
resolved "https://registry.yarnpkg.com/yallist/-/yallist-3.0.2.tgz#8452b4bb7e83c7c188d8041c1a837c773d6d8bb9"
yn@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/yn/-/yn-2.0.0.tgz#e5adabc8acf408f6385fc76495684c88e6af689a"
src/sdk/pynni/README.md 0 → 100644
View file @ 252f36f8
## Install ##
Install for current user:
pip install --user -e .
Install for all users:
pip install -e .
## Test ##
python setup.py test
src/sdk/pynni/nni/README.md 0 → 100644
View file @ 252f36f8
# How to use Tuner that NNI support?
For now, NNI could support tuner algorithm as following:
- TPE
- Random Search
- Anneal
- Naive Evolution
- ENAS (on going)
**1. Tuner algorithm introduction**
We will introduce some basic knowledge about tuner algorithm here. If you are an expert, you could skip this part and jump to how to use.
*1.1 TPE*
The Tree-structured Parzen Estimator (TPE) is a sequential model-based optimization (SMBO) approach. SMBO methods sequentially construct models to approximate the performance of hyperparameters based on historical measurements, and then subsequently choose new hyperparameters to test based on this model.
The TPE approach models P(x|y) and P(y) where x represents hyperparameters and y the associated evalate matric. P(x|y) is modeled by transforming the generative process of hyperparameters, replacing the distributions of the configuration prior with non-parametric densities. This optimization approach is described in detail in [Algorithms for Hyper-Parameter Optimization][1].
Comparing with other algorithm, TPE could be achieve better result when the number of trial experiment is small. Also TPE support continuous or discrete hyper-parameters. From a large amount of experiments, we could found that TPE is far better than Random Search.
*1.2 Random Search*
In [Random Search for Hyper-Parameter Optimization][2] show that Random Search might be surprsingly simple and effective. We suggests that we could use Random Search as basline when we have no knowledge about the prior distribution of hyper-parameters.
*1.3 Anneal*
*1.4 Naive Evolution*
Naive Evolution comes from [Large-Scale Evolution of Image Classifiers][3]. Naive Evolution requir more experiments to works, but it's very simple and easily to expand new features. There are some tips for user:
1) large initial population could avoid to fall into local optimum
2) use some strategies to keep the deversity of population could be better.
**2. How to use the tuner algorithm in NNI?**
User only need to do one thing: choose a Tuner```config.yaml```.
Here is an example:
```
# config.yaml
tuner:
tunerName: TPE
optimizationMode: Maximize
```
There are two filed you need to set:
```tunerName``` and ```optimizationMode```.
tunerName: TPE / Random / Anneal / Evolution
optimizationMode: Maximize / Minimize
[1]: https://papers.nips.cc/paper/4443-algorithms-for-hyper-parameter-optimization.pdf
[2]: http://www.jmlr.org/papers/volume13/bergstra12a/bergstra12a.pdf
[3]: https://arxiv.org/pdf/1703.01041.pdf
\ No newline at end of file
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment