10,622

Downloads

708

Downloads of v 2.0.5

17

Average downloads per day

2016-12-16

Last published

Share on Social Networks

Microsoft Azure HDInsight Job...

2.0.5

The HDInsight job management client.

To install Microsoft Azure HDInsight Job Management Library, run the following command in the Package Manager Console

Install-Package Microsoft.Azure.Management.HDInsight.Job

Requires NuGet 2.5 or higher.

Release Notes

2.0.5

  * Added support for running queries with large size.
  * Fixed network timeout error which occurs intermittently when HDinsight service is loaded.
 
2.0.4

  * Added support for new environments.
   
    The following API has changed
    public AzureStorageAccess(string storageAccountName, string storageAccountKey, string defaultStorageContainer, string storageAccountSuffix = null)
    Users can optionally specify the storage account URI suffix, for example : "core.chinacloudapi.cn"
   
    users can now use the GetJobOutput() and GetJobErrorLogs() APIs for the new environments

2.0.3

  * Added new API WaitForJobCompletion(string jobId, TimeSpan duration, TimeSpan waitInterval) to wait for completion of a job.

2.0.2
  * This package is now generally available.
 
2.0.1-preview

  * Task cancellation exception can come if http request doesn’t get finished before couple of minutes.

  * If empty job id is passed for Kill job operation then ArgumentException is raised instead of making a kill job request call.
 
  * Added Async APIs for submitting Jobs.

2.0.0-preview:

Some parameters of the job submission objects have changed:

  *  HiveJobSubmissionParameters
      1.  The parameter “Arguments” now has type IList<string>.
      2.  The parameter “Files” now has type IList<string>.
      3.  The parameter “Defines” now has type IDictionary<string, string>.
   
  *  MapReduceJobSubmissionParameters
      1.  The parameter “Arguments” now has type IList<string>.
      2.  The parameter “Files” now has type IList<string>.
      3.  The parameter “LibJars” now has type IList<string>.
      4.  The parameter “Defines” now has type IDictionary<string, string>.
      5.  The parameter “CmdEnv” now has type IDictionary<string, string>.
   
  *  MapReduceStreamingJobSubmissionParameters
      1.  The parameter “Arguments” now has type IList<string>.
      2.  The parameter “Defines” now has type IDictionary<string, string>.
      3.  There is a new parameter “Files” with type IDictionary<string, string>.
   
  *  PigJobSubmissionParameters
      1.  The parameter “Arguments” now has type IList<string>.
      2.  The parameter “Files” now has type IList<string>.
   
  *  SqoopJobSubmissionParameters
      1.  The parameter “Files” now has type IList<string>.
      2.  There is a new parameter “LibDir” with type string.
   
  *  You no longer need to provide a StatusDir argument for job submission requests – one is automatically generated for you.

  *  The unused parameter “UserName” has been removed for all job submission objects. The actual username is automatically picked up from BasicAuthenticationCloudCredentials.

  *  The unused parameter “errorLog” has been removed for all job submission objects. Task log collection is not supported in Hadoop 2.0 or greater. See JIRAs HIVE-6063 and HIVE-6671.

The methods “DownloadJobTaskLogsAsync”, “DownloadJobTaskLogs”, “GetJobTaskLogSummary”, and “GetJobTaskLogSummaryAsync” have been removed. Task log collection is not supported in Hadoop 2.0 or greater. See JIRAs HIVE-6063 and HIVE-6671.

The constructor for HDInsightJobManagementClient now supports providing an HTTP client retry policy for Job REST requests as an optional parameter.

  *  Example:
      public HDInsightJobManagementClient(string clusterDnsName, BasicAuthenticationCloudCredentials credentials, RetryPolicy retryPolicy = null)

  *  If a retry policy is not provided, then a default retry policy (HDInsightJobManagementClient.HDInsightRetryPolicy) will be used.

The methods “GetJobOutput”(+Async) and “GetJobErrorLogsAsync”(+Async) now accept an AzureStorageAccess object as input for storage details instead of three string arguments.

  *  Examples:
      Task<Stream> GetJobOutputAsync(string jobId, IStorageAccess storageAccess, CancellationToken cancellationToken);
      Stream GetJobOutput(this IJobOperations operations, string jobId, IStorageAccess storageAccess)
      Task<Stream> GetJobErrorLogsAsync(string jobId, IStorageAccess storageAccess, CancellationToken cancellationToken);
      Stream GetJobErrorLogs(this IJobOperations operations, string jobId, IStorageAccess storageAccess)

  *  Create the AzureStorageAccess object by providing Azure storage account details:
      new AzureStorageAccess(TestUtils.StorageAccountName, TestUtils.StorageAccountKey, TestUtils.DefaultContainer)

Owners

Authors

Microsoft

Copyright

Copyright © Microsoft Corporation

Tags

Dependencies

Version History

Version Downloads Last updated
Microsoft Azure... 2.0.4 1,258 Tuesday, November 1, 2016
Microsoft Azure... 2.0.3 2,669 Tuesday, June 7, 2016
Microsoft Azure... 2.0.2 263 Wednesday, May 4, 2016
Microsoft Azure... 2.0.1-preview 83 Wednesday, April 20, 2016
Microsoft Azure... 2.0.0-preview 924 Tuesday, March 15, 2016
Microsoft Azure... 1.1.0-preview 572 Friday, January 22, 2016
Microsoft Azure... 1.0.7-preview 961 Wednesday, October 21, 2015
Microsoft Azure... 1.0.6-preview 546 Wednesday, September 23, 2015
Microsoft Azure... 1.0.5-preview 863 Friday, September 4, 2015
Microsoft Azure... 1.0.4-preview 82 Thursday, September 3, 2015
Microsoft Azure... 1.0.3-preview 92 Tuesday, September 1, 2015
Microsoft Azure... 1.0.2-preview 87 Thursday, August 13, 2015
Microsoft Azure... 1.0.1-preview 1,514 Friday, July 31, 2015