ibm-information-center/dist/eclipse/plugins/i5OS.ic.apis_5.4.0.1/sslhands.htm

708 lines
19 KiB
HTML

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Copyright" content="Copyright (c) 2006 by IBM Corporation">
<title>SSL_Handshake()--Initiate the SSL Handshake Protocol</title>
<!-- All rights reserved. Licensed Materials Property of IBM -->
<!-- US Government Users Restricted Rights -->
<!-- Use, duplication or disclosure restricted by -->
<!-- GSA ADP Schedule Contract with IBM Corp. -->
<!-- Begin Header Records -->
<!-- Direct1 SCRIPT J converted by B2H R4.1 (346) (CMS) by V2KEA304 -->
<!-- at RCHVMW2 on 17 Feb 1999 at 11:05:09 -->
<!-- End Header Records -->
<link rel="stylesheet" type="text/css" href="../rzahg/ic.css">
</head>
<body>
<a name="Top_Of_Page"></a>
<!--Java sync-link-->
<script type="text/javascript" language="Javascript" src="../rzahg/synch.js">
</script>
<h2>SSL_Handshake()--Initiate the SSL Handshake Protocol</h2>
<div class="box" style="width: 60%;">
<br>
&nbsp;&nbsp;Syntax
<pre>
#include &lt;qsossl.h&gt;
int SSL_Handshake(SSLHandle* <em>handle</em>,
int <em>how</em>)
</pre>
<br>
&nbsp;&nbsp;Service Program Name: QSOSSLSR<br>
<!-- iddvc RMBR -->
<br>
&nbsp;&nbsp;Default Public Authority: *USE<br>
<!-- iddvc RMBR -->
<br>
&nbsp;&nbsp;Threadsafe: Yes<br>
<!-- iddvc RMBR -->
<br>
</div>
<p>The <strong><em>SSL_Handshake()</em></strong> function is used by a program
to initiate the SSL handshake protocol. Both the client and the server program
must call the SSL_Handshake verb in order to initiate the handshake
processing.</p>
<h3>Parameters</h3>
<dl>
<dt>SSLHandle* <strong><em>handle</em></strong>&nbsp; (input/output)</dt>
<dd>The pointer to an <strong><em>SSLHandle</em></strong> for an SSL session.
An <strong><em>SSLHandle</em></strong> is a typedef for a buffer of type struct
<strong><em>SSLHandleStr</em></strong>. In <strong>&lt;qsossl.h&gt;</strong>,
struct <strong><em>SSLHandleStr</em></strong> is defined as the following:<br>
<br>
<pre>
struct SSLHandleStr { /* SSLHandleStr */
int fd; /* Socket descriptor */
int createFlags; /* SSL_Create flags value */
unsigned protocol; /* SSL protocol version */
unsigned timeout; /* Timeout value in seconds */
unsigned char cipherKind[3]; /* Current 2.0 cipher suite */
unsigned short int cipherSuite; /* Current 3.0 cipher suite */
unsigned short int* cipherSuiteList; /* List of cipher suites */
unsigned int cipherSuiteListLen; /* Number of entries in
the cipher suites list */
unsigned char* peerCert; /* Peer certificate */
unsigned peerCertLen; /* Peer certificate length */
int peerCertValidateRc; /* Return code from
validation of certficate */
int (*exitPgm)(struct SSLHandleStr* sslh);
/* Authentication exit
program called when a
certificate is received
during SSL handshake */
};
</pre>
</dd>
</dl>
<p>The fields within the <strong><em>SSLHandle</em></strong> structure as
pointed to by <em>handle</em> are defined as follows:</p>
<table cellpadding="5">
<!-- cols="30 70" -->
<tr>
<td align="left" valign="top">int <strong><em>fd</em></strong>&nbsp;
(input)</td>
<td align="left" valign="top">The socket descriptor of the connection for which
the SSL handshake protocol is to be performed. This field was initialized by a
prior <em>SSL_Create()</em> API.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">int <strong><em>createFlags</em></strong>&nbsp;
(input)</td>
<td align="left" valign="top">Whether or not the SSL protocol is to be used. If
the field specifies a value that does not include the SSL_ENCRYPT flag, then
this function will return success without performing the SSL handshake
protocol. This field was initialized by a prior <em>SSL_Create()</em> API.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned int
<strong><em>protocol</em></strong>&nbsp; (input/output)</td>
<td align="left" valign="top">The type of SSL handshake protocol to be
performed. The protocol(s) that are acceptable as the handshake protocol for
this job. The following values may be specified for protocol and are defined in
<strong>&lt;qsossl.h&gt;</strong>.<br>
<br>
<pre>
SSL_VERSION_CURRENT 0 (TLS with SSL Version 3.0 and SSL
Version 2.0 compatibility)
SSL_VERSION_2 2 (SSL Version 2.0 only)
SSL_VERSION_3 3 (SSL Version 3.0 only)
TLS_VERSION_1 4 (TLS Version 1 only)
TLSV1_SSLV3 5 (TLS Version 1 with SSL
Version 3.0 compatibility)
</pre>
Upon return, this field will be set to reflect the protocol version actually
negotiated. If the <em>createFlags</em> field specifies a value that does not
include the SSL_ENCRYPT flag, then this field will be unchanged from its input
value.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned <strong><em>timeout</em></strong>&nbsp;
(input)</td>
<td align="left" valign="top">The approximate number of seconds to wait for the
SSL handshake protocol to complete. A value of 0 indicates to wait forever for
the handshake to complete.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned char
<strong><em>cipherKind[3]</em></strong>&nbsp; (output)</td>
<td align="left" valign="top">The cipher kind (which is the SSL Version 2.0
cipher suite) negotiated by the handshake.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned short int
<strong><em>cipherSuite</em></strong>&nbsp; (output)</td>
<td align="left" valign="top">The cipher suite type negotiated by the
handshake.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned short int*
<strong><em>cipherSuiteList</em></strong>&nbsp; (input)</td>
<td align="left" valign="top">A pointer to a cipher specification list that is
to be used during the handshake negotiation for this SSL session. This list is
a string of concatenated cipher specification values. Each cipher specification
is an unsigned short integer value. Any value provided will override, for this
SSL session, the default cipher specification list provided by a previous
<em>SSL_Init()</em> API or <em>SSL_Init_Application()</em> API . The valid
cipher suites allowed are defined in <strong>&lt;qsossl.h&gt;</strong>.&nbsp;&nbsp; A value of NULL
indicates one of the following:<br>
<br>
<ul>
<li>Use the cipher specification list provided by a previous
<em>SSL_Init()</em> API or <em>SSL_Init_Application()</em> API</li>
<li>Use the system default cipher specification list if the previous
<em>SSL_Init()</em> API or <em>SSL_Init_Application()</em> API did not provide
a cipher specification list</li>
</ul>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned int
<strong><em>cipherSuiteListLen</em></strong>&nbsp; (input)</td>
<td align="left" valign="top">The number of cipher suite entries specified in
the list pointed to by the <strong><em>cipherSuiteList</em></strong> field.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned char*
<strong><em>peerCert</em></strong>&nbsp; (output)</td>
<td align="left" valign="top">The pointer to the certificate received from the
peer system. For a client, this is a pointer to the server's certificate. For a
server with client authentication enabled, this is a pointer to the client's
certificate. For a server without client authentication this pointer value
remains unchanged.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">unsigned
<strong><em>peerCertLen</em></strong>&nbsp; (output)</td>
<td align="left" valign="top">The length of the certificate pointed to by the
<em>peerCert</em> field.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">int
(*<strong><em>exitPgm</em></strong>)(SSLHandle* sslh)&nbsp; (input)</td>
<td align="left" valign="top">A pointer to a user supplied function that is
called whenever a certificate is received during handshake processing. The
<em>exitPgm</em> will be passed the pointer to the
<strong><em>handle</em></strong>, which could include the peer's certificate.
The <em>exitPgm</em> returns a nonzero value if the peer's certificate is
accepted. The return of a zero value by the <em>exitPgm</em> will cause the
handshake processing to fail. Users of this function do not need to provide an
exit program. The pointer should be a NULL value if there is not a
user-supplied exit program. The exit program should be written in a threadsafe
manner.<br>
<br>
</td>
</tr>
<tr>
<td align="left" valign="top">int <strong><em>how</em></strong>&nbsp;
(input)</td>
<td align="left" valign="top">The type of SSL handshake to be performed. The
following values may be specified for handshake type and are defined in
<strong>&lt;qsossl.h&gt;</strong>.
<table cellpadding="5">
<tr>
<td align="left" valign="top">SSL_HANDSHAKE_AS_CLIENT (0) </td>
<td align="left" valign="top">Perform the handshake as a client.</td>
</tr>
<tr>
<td align="left" valign="top">SSL_HANDSHAKE_AS_SERVER (1) </td>
<td align="left" valign="top">Perform the handshake as a server.</td>
</tr>
<tr>
<td align="left" valign="top">SSL_HANDSHAKE_AS_SERVER_WITH_CLIENT_AUTH (2) </td>
<td align="left" valign="top">Perform the handshake as a server with client
authentication.</td>
</tr>
<tr>
<td align="left" valign="top">SSL_HANDSHAKE_AS_SERVER_WITH_OPTIONAL_CLIENT_AUTH (3) </td>
<td align="left" valign="top">Perform the handshake as a server with optional
client authentication.</td>
</tr>
</table>
</td>
</tr>
</table>
<br>
<br>
<h3>Authorities</h3>
<p>Authorization of *R (allow access to the object) to the key ring file and
its associated files is required.</p>
<br>
<h3>Return Value</h3>
<p>The <em>SSL_Handshake()</em> API returns an integer. Possible values
are:</p>
<dl>
<dt><em>[0]</em> </dt>
<dd>
<p>Successful return</p>
</dd>
<dt><em>[SSL_ERROR_BAD_CERTIFICATE]</em> </dt>
<dd>
<p>The certificate is bad.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_CERT_SIG]</em> </dt>
<dd>
<p>The certificate's signature is not valid.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_CERTIFICATE]</em> </dt>
<dd>
<p>The certificate is bad.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_CIPHER_SUITE]</em> </dt>
<dd>
<p>A cipher suite that is not valid was specified.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_MAC]</em> </dt>
<dd>
<p>A bad message authentication code was received.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_MALLOC]</em> </dt>
<dd>
<p>Unable to allocate storage required for SSL processing.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_MESSAGE]</em> </dt>
<dd>
<p>SSL received a badly formatted message.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_PEER]</em> </dt>
<dd>
<p>The peer system is not recognized.</p>
</dd>
<dt><em>[SSL_ERROR_BAD_STATE]</em> </dt>
<dd>
<p>SSL detected a bad state in the SSL session.</p>
</dd>
<dt><em>[SSL_ERROR_CERTIFICATE_REJECTED ]</em> </dt>
<dd>
<p>The certificate is not valid or was rejected by the exit program.</p>
</dd>
<dt><em>[SSL_ERROR_CERT_EXPIRED]</em> </dt>
<dd>
<p>The validity time period of the certificate is expired.</p>
</dd>
<dt><em>[SSL_ERROR_CLOSED]</em> </dt>
<dd>
<p>The SSL session ended.</p>
</dd>
<dt><em>[SSL_ERROR_IO]</em> </dt>
<dd>
<p>An error occurred in SSL processing; check the <em>errno</em> value.</p>
</dd>
<dt><em>[SSL_ERROR_NO_CERTIFICATE]</em> </dt>
<dd>
<p>No certificate is available for SSL processing.</p>
</dd>
<dt><em>[SSL_ERROR_NO_CIPHERS]</em> </dt>
<dd>
<p>No ciphers available or specified.</p>
</dd>
<dt><em>[SSL_ERROR_NO_INIT]</em> </dt>
<dd>
<p>SSL_Init() was not previously called for this job.</p>
</dd>
<dt><em>[SSL_ERROR_NOT_TRUSTED_ROOT]</em> </dt>
<dd>
<p>The certificate is not signed by a trusted certificate authority.</p>
</dd>
<dt><em>[SSL_ERROR_PERMISSION_DENIED]</em> </dt>
<dd>
<p>Permission was denied to access object.</p>
</dd>
<dt><em>[SSL_ERROR_SSL_NOT_AVAILABLE]</em> </dt>
<dd>
<p>SSL is not available for use.</p>
</dd>
<dt><em>[SSL_ERROR_UNKNOWN]</em> </dt>
<dd>
<p>An unknown or unexpected error occurred during SSL processing.</p>
</dd>
<dt><em>[SSL_ERROR_UNSUPPORTED_CERTIFICATE_TYPE]</em> </dt>
<dd>
<p>i5/OS does not support the certificate's type.</p>
</dd>
<dt><em>[SSL_ERROR_UNSUPPORTED_CERTIFICATE_TYPE]</em> </dt>
<dd>
<p>i5/OS does not support the certificate's type. </p>
</dd>
</dl>
<br>
<br>
<h3>Error Conditions</h3>
<p>When the <em>SSL_Handshake()</em> API fails with a return code of
[SSL_ERROR_IO], <em>errno</em> can be set to:</p>
<dl>
<dt><em>[EACCES]</em> </dt>
<dd>
<p>Permission denied. </p>
</dd>
<dt><em>[EBADF]</em> </dt>
<dd>
<p>Descriptor not valid. </p>
</dd>
<dt><em>[EBUSY]</em> </dt>
<dd>
<p>Resource busy. </p>
</dd>
<dt><em>[ECONNRESET]</em> </dt>
<dd>
<p>A connection with a remote socket was reset by that socket. </p>
</dd>
<dt><em>[EDEADLK]</em> </dt>
<dd>
<p>Resource deadlock avoided. </p>
</dd>
<dt><em>[EFAULT]</em> </dt>
<dd>
<p>Bad address.</p>
<p>The system detected an address that was not valid while attempting to access
the <em>handle</em> parameter or one of the address fields in the
<em>handle</em> parameter.</p>
</dd>
<dt><em>[EINTR]</em> </dt>
<dd>
<p>Interrupted function call. </p>
</dd>
<dt><em>[EINVAL]</em> </dt>
<dd>
<p>Parameter not valid.</p>
<p>This error code indicates one of the following:</p>
<ul>
<li>The <em>socket_descriptor</em> type is not SOCK_STREAM or address family is
not AF_INET or AF_INET6.</li>
<li>One of the parameters passed is not valid or is NULL.</li>
<li>The <em>protocol</em> field contains a value that is not valid.</li>
</ul><br>
</dd>
<dt><em>[EALREADY]</em> </dt>
<dd>
<p>Operation already in progress.</p>
<p>An <em>SSL_Handshake()</em> API has already been previously successfully
completed.</p>
</dd>
<dt><em>[EIO]</em> </dt>
<dd>
<p>Input/output error. </p>
</dd>
<dt><em>[ENOBUFS]</em> </dt>
<dd>
<p>There is not enough buffer space for the requested operation. </p>
</dd>
<dt><em>[ENOTCONN]</em> </dt>
<dd>
<p>Requested operation requires a connection.</p>
<p>This error code indicates one of the following:</p>
<ul>
<li>The <em>socket_descriptor</em> is not for a socket that is in a connected
state.</li>
<li>The <em>socket_descriptor</em> has not had SSL support enabled.</li>
</ul><br>
</dd>
<dt><em>[ENOTSOCK]</em> </dt>
<dd>
<p>The specified descriptor does not reference a socket. </p>
</dd>
<dt><em>[EPIPE]</em> </dt>
<dd>
<p>Broken pipe. </p>
</dd>
<dt><em>[ETIMEDOUT]</em> </dt>
<dd>
<p>A remote host did not respond within the timeout period. </p>
</dd>
<dt><em>[EUNATCH]</em> </dt>
<dd>
<p>The protocol required to support the specified address family is not
available at this time. </p>
</dd>
<dt><em>[EUNKNOWN]</em> </dt>
<dd>
<p>Unknown system state. </p>
</dd>
</dl>
<p>Any <em>errno</em> that can be returned by <strong>send()</strong> or
<strong>recv()</strong> can be returned by this API. See <a href=
"unix8.htm">Sockets APIs</a> for a description of the <em>errno</em> values
they return.</p>
<p>If an <em>errno</em> is returned that is not in this list, see <a href=
"unix14.htm">Errno Values for UNIX-Type Functions</a> for a description of the
<em>errno</em>.</p>
<br>
<br>
<h3>Error Messages</h3>
<table width="100%" cellpadding="5">
<!-- cols="15 85" -->
<tr>
<th align="left" valign="top">Message ID</th>
<th align="left" valign="top">Error Message Text</th>
</tr>
<tr>
<td align="left" valign="top">CPE3418 E</td>
<td align="left" valign="top">Possible APAR condition or hardware failure.</td>
</tr>
<tr>
<td align="left" valign="top">CPF9872 E</td>
<td align="left" valign="top">Program or service program &amp;1 in library
&amp;2 ended. Reason code &amp;3.</td>
</tr>
<tr>
<td align="left" valign="top">CPFA081 E</td>
<td align="left" valign="top">Unable to set return value or error code.</td>
</tr>
</table>
<br>
<br>
<h3>Usage Notes</h3>
<ol>
<li>The <em>SSL_Handshake()</em> function is only valid on sockets that have an
address family of <samp>AF_INET</samp> or <samp>AF_INET6</samp> and a socket type of <samp>SOCK_STREAM</samp>. If the descriptor
pointed to by the <em>handle</em> structure parameter does not have the correct
address family and socket type, [SSL_ERROR_IO] is returned and the
<em>errno</em> value is set to EINVAL.<br>
<br>
</li>
<li>The <em>SSL_Handshake()</em> function can be called only one time per SSL
session. If a secondary call of <em>SSL_Handshake()</em> occurs within the same
established SSL session, then it will fail and the <em>errno</em> will be set
to <samp>[einval]</samp>.<br>
<br>
</li>
<li>A successful <em>SSL_Init()</em> or or <em>SSL_Init_Application()</em> API
and a successful <em>SSL_Create()</em> API must be called prior to an
<em>SSL_Handshake()</em> API. The <em>SSL_Init()</em> API or
<em>SSL_Init_Application()</em> API is used to establish a certificate and
private key for either of the following:<br>
<br>
<ul>
<li>A successful handshake as a server</li>
<li>A successful handshake as a client when connected to a server performing
client authentication</li>
</ul>
<br>
</li>
<li>The <em>SSL_Create()</em> API is used to enable
SSL support for the specified socket descriptor.<br>
<br></li>
<li>When doing <em>SSL_Handshake()</em> with a <em>how</em> parameter value
of <samp>SSL_HANDSHAKE_AS_SERVER</samp>,
<samp>SSL_HANDSHAKE_AS_SERVER_WITH_CLIENT_AUTH</samp>, or
<samp>SSL_HANDSHAKE_AS_SERVER_WITH_OPTIONAL_CLIENT_AUTH</samp>, the
<em>cipherSuite</em> value (if <samp>TLS_VERSION_1</samp> or
<samp>SSL_VERSION_3</samp> protocol) or the <em>cipherKind</em> (if
<samp>SSL_VERSION_2</samp> protocol) will be the first cipher found in
the ordered <em>cipherSuiteList</em> list that was also found in the cipher list
provided by the client during the SSL handshake.<br>
<br>
</li>
<li>When doing <em>SSL_Handshake()</em> with a <em>how</em> parameter value
of <samp>SSL_HANDSHAKE_AS_CLIENT</samp>, the cipher specification list will be
sent to the server in the client hello in the order found in the
<em>cipherSuiteList</em>, however the value from that list that is
negotiated for the <em>cipherSuite</em> value (if <samp>TLS_VERSION_1</samp> or
<samp>SSL_VERSION_3</samp> protocol) or the
<em>cipherKind</em> (if <samp>SSL_VERSION_2</samp> protocol) is determined by
the server policy.
<br>
</li>
</ol>
<br>
<h3>Related Information</h3>
<ul>
<li><a href="sslcreat.htm">SSL_Create()</a>--Enable SSL Support for the Specified
Socket Descriptor<br><br></li>
<li><a href="ssldest.htm">SSL_Destroy()</a>--End SSL Support for the Specified SSL
Session<br><br></li>
<li><a href="sslinit.htm">SSL_Init()</a>--Initialize the Current Job for
SSL<br><br></li>
<li><a href="sslread.htm">SSL_Read()</a>--Receive Data from an SSL-Enabled Socket
Descriptor<br><br></li>
<li><a href="sslread.htm">SSL_Write()</a>--Write Data to an SSL-Enabled Socket
Descriptor</li>
</ul>
<br>
<hr>
API introduced: V4R3
<hr>
<center>
<table cellpadding="2" cellspacing="2">
<tr align="center">
<td valign="middle" align="center"><a href="#Top_Of_Page">Top</a> | <a href=
"unix.htm">UNIX-Type APIs</a> | <a href="aplist.htm">APIs by category</a> </td>
</tr>
</table></center>
</body>
</html>