infiniband-hca_ddk Mailing List for Linux InfiniBand Project

> The open_ca() flow. Upon _init() of the AL dynamic library linked with the
> user process (lately, we discussed moving that to ib_open_al()), UAL
> performs the ual_open_ca() ioctl that allocates the kernel context for the
> process. This call is "padded" with the pre_ and post_ UVP calls with the
> regular semantics, where the vendor-specific user context for the CA can
> be established. However, the KAL code that serves the ioctl does not
> provide a hook to the KVP that could be used for setting up the CA
> resources for the user context in the kernel. Instead, the AL expects the
> kernel to perform the vendor specific resource setup upon the PD
> allocation. 
> Therefore, every KVP that has a notion of per-CA-per-process resources is
> forced to defer the setup of those to the first PD allocation, which is
> tricky, requires extra synchronization, and overloads the ib_alloc_pd()
> call with non-natural semantics. Moreover, we are not sure that we can
> support this for long in the new FW versions and HW revisions. The other
> solutions, like performing an ioctl to the KVP from the post_ call, are
> even more hacky. 
> The funny thing is that your design has gone more than half-way in
> defining the user-space plugins for the UVP, so you do acknowledge that
> the user process might want to setup some context with the first
> open_ca(). However, we have no hook for the KVP in this process. It will
> be more than valuable if you extended the KVP API with this one call.
> 
> --------------------------------------------------------------------------
> -----------------
> Edward Bortnikov                         mailto:ed...@me...
> Mellanox Technologies Ltd           +972-4-9097200 (ext. 334)
> 
> 

The Alpha 1 release for the Linux InfiniBand Project is now posted in =
the project's BitKeeper repository (http://infiniband.bkbits.net/).  =
Refer to label "alpha_release_1".  The release consists of:

*	Kernel mode components: IB Access Layer, IPoIB, SDP (the Mellanox HCA =
VPD must be obtained directly from Mellanox)
*	User mode components: uDAPL, OpenSM
*	Configuration: IA-32, SMP, debug builds only
*	Basic functionality only - performance characterization and tuning =
still needs to be undertaken

Refer to the release notes contained in BitKeeper for further details.

Bugs should be submitted to =
http://sourceforge.net/tracker/?group_id=3D52127&atid=3D465806.

Upcoming releases will focus on delivering performance enhancements, a =
user mode VPD for the Mellanox HCA and a port of the stack to the =
Itanium Processor Family (IPF).

   *  Dave

Dear Lists' members,

 I am wondering what component should do serialization for simlutaneous 
or concurrent accesses on various handles such as cq_h, qp_h etc. in
general.

 I have made a cursory look into Early reference code HCA driver for
Intel's HCA and the Access Layer, and I have not found  serialization
code forvarious types of accesses on handles. For example, what
components should do what to prevent inconsistency among pieces of
information if destory_qp() is called while modify_qp() is ongoing ?

 Could someone give me some basic and/or just rough guidance
statements on the serialization policy of handles ? I am wondering if
all the serialzation should be done by HCA drivers (in the callbacks
they provide), while it would probably make HCA drivers rather complex.

Regards,
--
HASHIMOTO Tsuyoshi (HASHIMOTO is my family name.)

Please note SourceForge changeset version 1.96 is required for
sdk006_ib196 of Mellanox drivers.
=20
SourceForge CS version 1.96 is now been tagged as alpha1_rc1 and can be
cloned using the tag name as shown below.

=20
bk clone -ralpha1_rc1 source_repository_name
=20
-Anil

	-----Original Message-----
	From: Chanan Shamir [mailto:Ch...@Me...]=20
	Sent: Monday, June 02, 2003 4:21 PM
	To: inf...@li...
	Cc: 'inf...@li...'
	Subject: [Infiniband-general] Updated Mellanox VPD
=09
=09


	An updated Mellanox driver and release notes - tagged as
sdk006_ib196 are available from Mellanox website at
http://docs.mellanox.com under CodeReleases/InfiniHost/SourceForge.

	As the tag indicates this driver is targeted for:=20
	        Mellaonx SDK 0.0.6 (and FW 1.16.0000)=20
	        SF Code version 1.9 6 =20

	This code drop fixes a race condition on driver unload and
improves readability of error messages by including a text description
of error codes.

	Chanan=20

An updated Mellanox driver and release notes - tagged as sdk006_ib196 are
available from Mellanox website at http://docs.mellanox.com under
CodeReleases/InfiniHost/SourceForge.

As the tag indicates this driver is targeted for:
	Mellaonx SDK 0.0.6 (and FW 1.16.0000)
	SF Code version 1.94

This code drop fixes a race condition on driver unload and improves
readability of error messages by including a text description of error
codes.

Chanan

Hi Tsuyoshi,

The verb spec mentions max limits to be specified in the hca attributes
for quantities such as 

- maximum number of regions supported
- maximum size of regions
- maximum number of windows supported etc.

It was our belief that rather than specifying maximums, most HCA vendors
would start with some initial limit for those and be able to expand or
grow as need arises. Hence the name "init_*" (this thing happened so
long back and had to jog memory to get those out.., we will update the
headers on next chance.)

Av_port_check - indicates capability by HCA to perform port checks for
address vectors specified for use in UD qp's data transfer. (check vol
1.1 chapter 11.2.1.2 around line 36 under "HCA specific values".

Num_page_sizes indicate support for multiple page size support for
physical pages. The number of pages supported is a variable size array
in ca_attr structure.

When using physical page register, user can pass page_size in
ib_phys_create_t. This will help conserve use fewer TPT resources and
help performance in doing lookups too.

Hope this helps.
If I missed something, please let me know

ashokr


-----Original Message-----
From: HASHIMOTO, Tsuyoshi [mailto:ts...@cs...] 
Sent: Wednesday, May 14, 2003 12:52 AM
To: inf...@li...
Subject: [Infiniband-hca_ddk] What do these fields of ib_ca_attr_t mean
?


Dear List members,

I am a member of Fujitsu HCA driver development team, and I am wondering
what the following members of ib_ca_attr_t defined in
iba/base/shared/include/iba/ib_types.h without comments, and it seems
difficult to find their counterparts in the IBA spec. and/or 
HCA profile documentation found on http://www.infinibandta.org/.

Could someone explain the meaning of them, and hopefully suggest what
kind of values should be set by HCA driver ?

	uint32_t				max_pds;
// What should be set if HCA does not impose any particular limit ? //
Will any large enough values (fit in this field) do, perhaps? 

	uint32_t				init_regions;
	uint64_t				init_region_size;
	uint32_t				init_windows;
// "init" does not seem to suggest what values should be set. // Are
these fields might have some meaning other HCAs like // Intel's gen1 HCA
and/or Mellanox (profile B) HCA ?

	uint32_t				max_addr_handles;
// What should be set if HCA does not impose any particular limit ? //
Will any large enough values do (fit in this field), perhaps? 

	boolean_t				av_port_check;
// Does av_port_check indicate some relation between address vector 
// and and CA's port ? If so, what kind of relation ? And what does //
this field mean, anyway?

	uint32_t				num_page_sizes;
// I do not see what num_page_sizes mean. Why this should be
// an attribute of an HCA ?

Regards,
--
HASHIMOTO Tsuyoshi (HASHIMOTO is my family name.)


-------------------------------------------------------
Enterprise Linux Forum Conference & Expo, June 4-6, 2003, Santa Clara
The only event dedicated to issues related to Linux enterprise solutions
www.enterpriselinuxforum.com

_______________________________________________
Infiniband-hca_ddk mailing list Inf...@li...
https://lists.sourceforge.net/lists/listinfo/infiniband-hca_ddk

Dear list members,

 I assume complib can be considered as a part of HCA DDK  (i.e. meant
to be used in HCA drivers), and I suppose "what contexts of drivers (or
rather in kernel) are allowed to call each of them" might preferably
be remarked in complib documentation (source code comments are thought
as a part of documentation). The rationale follows.

 For example, suppose a certain CL function uses kmalloc() with
relatively low priority of GFP_KERNEL, and so the CL function might
sleep. If so, the CL function should not be called in interrupt
contexts, however, this kind of limitation does not seem to be 
remarked in the current version of CL documentation.

 This lack of information seems likely to cause problems in drivers
or any other kernel components which might use CL functions, and
might discourage the use of CL functions. I do not think this is
what CL implementors wish to happen. From this point of view, each
CL function description should include a remark on the context where
it can be called. And I suppose this might be useful as a reminder for
CL function implementors, too.

 For example, cl_qcpool_get() seems to do this kind of low priority
memory allocation via cl_qcpool_grow() unless p_pool->grow_size is 0.
 And so a remark "not to use cl_qcpool_get() (and any other wrappers
for this) in interrupt context without setting p_pool->grow_size to 0",
or some equivalent remarks might prevent possible bugs in drivers.

 BTW, DDKs of commercial Unices such as Solaris seem to have this kind
of remarks, and so such remarks might help relatively new Linux driver
writers who has experiences in some other Unices than Linux.

Regards,
--
HASHIMOTO Tsuyoshi (HASHIMOTO is my family name.)

Dear List members,

I am a member of Fujitsu HCA driver development team, and I am
wondering what the following members of ib_ca_attr_t defined in
iba/base/shared/include/iba/ib_types.h without comments, and
it seems difficult to find their counterparts in the IBA spec. and/or 
HCA profile documentation found on http://www.infinibandta.org/.

Could someone explain the meaning of them, and hopefully suggest what
kind of values should be set by HCA driver ?

	uint32_t				max_pds;
// What should be set if HCA does not impose any particular limit ?
// Will any large enough values (fit in this field) do, perhaps? 

	uint32_t				init_regions;
	uint64_t				init_region_size;
	uint32_t				init_windows;
// "init" does not seem to suggest what values should be set.
// Are these fields might have some meaning other HCAs like
// Intel's gen1 HCA and/or Mellanox (profile B) HCA ?

	uint32_t				max_addr_handles;
// What should be set if HCA does not impose any particular limit ?
// Will any large enough values do (fit in this field), perhaps? 

	boolean_t				av_port_check;
// Does av_port_check indicate some relation between address vector 
// and and CA's port ? If so, what kind of relation ? And what does
// this field mean, anyway?

	uint32_t				num_page_sizes;
// I do not see what num_page_sizes mean. Why this should be
// an attribute of an HCA ?

Regards,
--
HASHIMOTO Tsuyoshi (HASHIMOTO is my family name.)

SDP has now been placed in the BitKeeper source repository. 

At this time, SDP has only been validated on Mellanox Profile B hardware.
Besides this hardware, you will need the latest sources from
<http://infiniband.bkbits.net> and the latest drivers and firmware from
Mellanox. 

Also, the patches necessary for 2.4.18 based kernels is placed now in
BitKeeper for both IPoIB, and for OPS. For transparent offload to SDP for
legacy TCP applications, a patch to socket.{ch} is also available in
BitKeeper under the directory base/linux/patches.

Also, several changes were made to the home page
(http://infiniband.sourceforge.net/):
*  Redirected the existing download links (for Access Layer, OpenSM, IPoIB
and SRP) to point to the BitKeeper repository now that CVS is no longer used
for this project
*  Updated the download link for SDP to point to BitKeeper now that SDP
sources are available
*  Posted updated draft (version 1.3) of the Verb Implementer's Guide (VIG)
*  Updated the contact information for the OpenSM

   *  Dave

Hello

Some more details.

If you are using proxy, then set the following environment variables....

If you are using bash, 

export HTTP_PROXY_PORT=<myproxyport>
export HTTP_PROXY_HOST=<myproxyhost>


for developers who need to push changes, you need ssh(1) access.

Please send your public key file as an attachment.(only if you plan to make
changes to the source base, you don't need to have ssh keys to just download
the tree)

Again if you are behind a firewall, and your proxy permits tunneling ssh
sessions, then download corkscrew http://www.agroman.net/corkscrew/ and add
a line to your $HOME/.ssh/config as follows...

bkbits.net
	ProxyCommand /usr/local/bin/corkscrew <my_proxy_host_name>
<my_proxy_port_number>  %h %p
(second line is broken due to mailer issues, you need to have it in a single
line)


Build instructions can be found under base/linux/Documentation area.

Cheers,
ashokr

-----Original Message-----
From: Lomartire, David M [mailto:dav...@in...] 
Sent: Friday, February 14, 2003 4:01 PM
To: InfiniBand Access Layer; InfiniBand Early_Reference; InfiniBand General;
InfiniBand HCA_DDK; InfiniBand IPoIB; InfiniBand SDP; InfiniBand SRP;
InfiniBand Subnet Management
Subject: [Infiniband-early_ref] Linux InfiniBand Project Sources now in
BitKeeper!

No more tarballs for the InfiniBand source drops!

Now the InfiniBand sources are being maintained in a BitKeeper repository.

The location of the repository is http://infiniband.bkbits.net/.

You can obtain a copy of BitKeeper for a wide variety of platforms from
http://www.bitkeeper.com/Products.Downloads.html.

After installing BitKeeper, to download the sources, simply issue the
following command:
      bk clone http://infiniband.bkbits.net/iba  iba 

More detailed information on how to work with a BitKeeper repository can be
found at http://www.bitkeeper.com/Hosted.html.

Please contact your friendly neighborhood Project Administrator with any
questions.

   *  Dave



-------------------------------------------------------
This SF.NET email is sponsored by: FREE  SSL Guide from Thawte
are you planning your Web Server Security? Click here to get a FREE
Thawte SSL guide and find the answers to all your  SSL security issues.
http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en
_______________________________________________
Infiniband-early_ref mailing list
Inf...@li...
https://lists.sourceforge.net/lists/listinfo/infiniband-early_ref

No more tarballs for the InfiniBand source drops!

Now the InfiniBand sources are being maintained in a BitKeeper repository.

The location of the repository is http://infiniband.bkbits.net/.

You can obtain a copy of BitKeeper for a wide variety of platforms from
http://www.bitkeeper.com/Products.Downloads.html.

After installing BitKeeper, to download the sources, simply issue the
following command:
      bk clone http://infiniband.bkbits.net/iba  iba 

More detailed information on how to work with a BitKeeper repository can be
found at http://www.bitkeeper.com/Hosted.html.

Please contact your friendly neighborhood Project Administrator with any
questions.

   *  Dave

Now posted in the "InfiniBand" files area
(http://sourceforge.net/project/showfiles.php?group_id=52127) is the 
latest drop of the SourceForge Linux InfiniBand Project stack. 

With this drop, IPoIB is now much more stable. Improvements include: 
- IOMeter run successfully for 48 hours 
- Completed overnight runs with IPoIB in a 6 node cluster 
- Enabled multicast support across multiple switch fabrics 

See additional details in the release notes located in the file release
area.

At present, the stack only runs via the HCA driver supplied by Mellanox.
Note: At this time the Mellanox driver is obtained from Mellanox directly
via their DDK and is not posted to this project.  Refer to the Mellanox web
site at 
http://docs.mellanox.com under CodeRelease/InfiniHost/SourceForge. 

Note: This code is not yet in the project CVS repository! Once CVS has been
updated, an announcement will be sent out. So, for now, the only place to
get this latest code is from the tarball.

   *  Dave

Now posted in the "InfiniBand" files area
(http://sourceforge.net/project/showfiles.php?group_id=52127) is the latest
drop of the SourceForge InfiniBand project stack.  With this drop, IPoIB is
now functional with the Profile B HCA from Mellanox.  (At present, the stack
only runs via the HCA driver supplied by Mellanox.)  Note: At this time the
Mellanox driver is obtained from Mellanox directly via their DDK (and is not
posted to this project).

Other items of interest in this drop include:

*  Multicast support (in both IPoIB and OpenSM) - single switch fabrics only
at this time
*  CM support (both kernel and usermode)
*  OpenSM supports standard SA queries (see the "Subnet Management"
sub-project at http://infiniband.sourceforge.net/ for a complete list of the
queries supported)
*  Complete SMI/GSI implementation (GSI now includes support for RMPP)
*  OpenSM generates a subnet config file which can be read by the Mellanox
GUI in order to display the fabric 

See additional details in the release notes posted with the drop.

Note: This code is not yet in the project CVS repository! Once CVS has been
updated, an announcement will be sent out. So, for now, the only place to
get this latest code is from the tarball.

   *  Dave

WE are currently integrating with the latest source drop from Mellanox
internally. WE will perform additional integration testing with the
Communication Manager (CM) and the Mellanox stack that supports RC.

If all goes well we will have another drop of the ib stack this week.

-----Original Message-----
From: Masao Uebayashi [mailto:ueb...@pu...] 
Sent: Friday, November 29, 2002 2:48 AM
To: dav...@in...
Cc: inf...@li...;
inf...@li...
Subject: Re: [Infiniband-hca_ddk] First drop of operational "Profile B"
stack

Does anyone have any clue about when RC, RD, and UC transport will be
supported by Mellanox HCA driver?

Masao


-------------------------------------------------------
This SF.net email is sponsored by: Get the new Palm Tungsten T 
handheld. Power & Color in a compact size! 
http://ads.sourceforge.net/cgi-bin/redirect.pl?palm0002en
_______________________________________________
Infiniband-hca_ddk mailing list
Inf...@li...
https://lists.sourceforge.net/lists/listinfo/infiniband-hca_ddk

Does anyone have any clue about when RC, RD, and UC transport will be
supported by Mellanox HCA driver?

Masao

Summary: With this drop, the opensm is now functional with the Profile =
B HCA
from Mellanox.
      *  opensm can configure a point-to-point fabric (without a =
switch) via
the Access Layer and the HCA driver supplied by Mellanox.=20
      *  Note: At this time the Mellanox driver is obtained from =
Mellanox
directly via their DDK (and is not posted to this project).=20

Other items of note include:=20
- Complete SMI/GSI implementation in AL=20
- Multicast functions implemented on AL (SA multicast support is being
tested)=20
- First code drop for SRP (Code Complete, but not tested)=20

What to expect in the next drop:=20
- Functional Connection Manager=20
- Multicast Functions=20
- Path Query functions=20
- Support for profile B switch=20

See additional details in both sets of release notes located in the =
file
release area at
<http://sourceforge.net/project/showfiles.php?group_id=3D52127>.=20

Note: This code is not yet in the project CVS repository! When CVS has =
been
updated, an announcement will be sent out. So, for now, the only place =
to
get this latest code is from the tarball posted at the file list at
<http://sourceforge.net/project/showfiles.php?group_id=3D52127>.

   =B7  Dave

This drop is comprised of two tarballs:=20

* iba_020910.tgz=20
- Added a shim driver that works with the Intel=AE 82808BA verbs driver =
and
the Intel=AE 82808BA hardware=20
- AL contains additional verbs support=20
- AL now supports user mode=20
- First release of the AL Test Suite (alts). Refer to Readme.txt in
linux\drivers\iba\alts or linuxuser\iba\alts for more details.=20
- Feature complete drop of ipiob. Refer to Readme in
linux\drivers\iba\net\ipoib for more details.=20

* mellanox_020909.tgz=20
This is the first code release of Mellanox HCA support for the =
InfiniBand
SourceForge project. Instructions for installation, building and =
executing
the code and related test(s) are provided in the Release Notes. This =
release
supports the UD transport in kernel mode.=20

See additional details in both sets of release notes located in the =
file
release area at =
http://sourceforge.net/project/showfiles.php?group_id=3D52127.


Note: This code is not yet in the project CVS repository! Once CVS has =
been
updated, an announcement will be sent out. So, for now, the only place =
to
get this latest code is from the tarball posted at the file list at
http://sourceforge.net/project/showfiles.php?group_id=3D52127.

   *  Dave

I just wanted to make sure everyone was aware of the recent documentation
drops which have been placed out on the Linux InfiniBand Project home page.

New documents include:
*	Access Layer User's Guide
*	Access Layer High Level Design
*	SRP High Level Design
*	"OpenSM" API document and architecture presentation

Updated documents include:
*	Linux Software Architecture Specification (SAS) which supersedes
many of the SAS documents which were previously posted in the individual
sub-project areas
*	SDP High Level Design and SDP API documentation
*	Verbs API, Usermode Verbs API, Access Layer API and Component
Library API documentation which matches the most recent code drop in the
tarball on 7/16.

Refer to the various sub-projects on http://infiniband.sourceforge.net for
more details.  Note that all changes to any area of the home page can be
found at the "Recent Changes" link.

   *  Dave

Hi Chanan.
 
We went throught the API's when the shim for gen1 was being developed and
fixed most of those cases. The CVS tree is not updated yet. 
 
could you  take a look at what is there in the most recent tarball to see if
such cases exist.
 
if they still exist in the tarball, let me know the specific API's and we
can get them fixed.
 
ashokr
-----Original Message-----
From: Tillier, Fabian [mailto:fab...@in...]
Sent: Monday, August 05, 2002 3:16 PM
To: 'Chanan Shamir'; 'inf...@li...'
Subject: RE: [Infiniband-hca_ddk] (no subject)



OUT type* const name should allow changing the value pointed to, but not
where the pointer points.  This should work for out parameters.  Note that
this syntax is different than OUT type const *name (the location of the *
with respect to the const).

 

OUT const type* name is an error as you noticed, and should be changed to
the above syntax or be removed all together.

 

- Fab

 

-----Original Message-----
From: Chanan Shamir [mailto:Ch...@Me...] 
Sent: Monday, August 05, 2002 2:59 PM
To: 'inf...@li...'
Subject: [Infiniband-hca_ddk] (no subject)

 

There are several places in include/iba/ib_ci.h where function arguments are
defined as 
        "OUT const type *name". 
This causes the following compiler warning when assigning through these
pointers: 
        warning: assignment of read-only location. 

There seems to be no difference in this regard, between: 
        OUT const type *name 
and 
        OUT type const *name 
Both of which are used in header definitons. 

The 'const' modifier should be removed from all parameter definitions, 
which are OUT or "IN OUT". This may apply to other header files as well. 

Chanan 

OUT type* const name should allow changing the value pointed to, but not
where the pointer points.  This should work for out parameters.  Note that
this syntax is different than OUT type const *name (the location of the *
with respect to the const).

 

OUT const type* name is an error as you noticed, and should be changed to
the above syntax or be removed all together.

 

- Fab

 

-----Original Message-----
From: Chanan Shamir [mailto:Ch...@Me...] 
Sent: Monday, August 05, 2002 2:59 PM
To: 'inf...@li...'
Subject: [Infiniband-hca_ddk] (no subject)

 

There are several places in include/iba/ib_ci.h where function arguments are
defined as 
        "OUT const type *name". 
This causes the following compiler warning when assigning through these
pointers: 
        warning: assignment of read-only location. 

There seems to be no difference in this regard, between: 
        OUT const type *name 
and 
        OUT type const *name 
Both of which are used in header definitons. 

The 'const' modifier should be removed from all parameter definitions, 
which are OUT or "IN OUT". This may apply to other header files as well. 

Chanan 

There are several places in include/iba/ib_ci.h where function arguments are
defined as
	"OUT const type *name".
This causes the following compiler warning when assigning through these
pointers:
	warning: assignment of read-only location.

There seems to be no difference in this regard, between:
	OUT const type *name
and
	OUT type const *name
Both of which are used in header definitons.

The 'const' modifier should be removed from all parameter definitions,
which are OUT or "IN OUT". This may apply to other header files as well.

Chanan

The main problem is that while a QP is in the timewait state, none of the
resources associated with the QP can be released to verbs.  This is
independent of whether the user has called cm_dreq().

For example, after a QP has been disconnected, it is placed into the
timewait state.  While in the timewait state, it cannot be released to
verbs, or it risks verbs re-allocating the QP.  This prevents the associated
PD and CQs from being released as well.

The result is that a client cannot simply release the AL, but instead must
wait for all QPs to exit the timewait state.  Moving timewait beneath verbs
eliminates the problem of having to keep resources hanging around in the AL
after they are no longer needed by a client.  And the code to handle
timewait beneath verbs is substantially easier than the code necessary to
handle it above.

- Sean



-----Original Message-----
From: Joseph Golio [mailto:go...@vi...]
Sent: Thursday, July 18, 2002 8:26 AM
To: Raj, Ashok
Cc: 'inf...@li...';
'inf...@li...'; 'Ke...@Me...';
'vu...@me...'; 'Ch...@me...'; Coffman, Jerrie L; Hefty,
Sean; Tillier, Fabian; Narayanan, Chandramouli; Nair, Sandeep R; Pinto,
Oscar; 'inf...@li...'; Woodruff, Robert J
Subject: Re: [Infiniband-general] Handling of timed wait state for QP/EE


"Raj, Ashok" wrote:

All,


> Hello
>
> When designing AL, the development team ran into one area that would help
if
> we shift the responsibility of where the timedwait state is managed for
> QP/EE resources. Some thoughts and solutions are outlined here. Would like
> to hear what others think about this and other ideas.
>
> The Infiniband spec says its the Communication Manager issue. The
> implementation issue is that when AL consumers release all resources
> this would mean that it should keep the QP/EE resource around until the
> TIMEWAIT state is expired. THis would mean that the application would have
a
> long wait since the ib_close_al() will not succeed waiting for the
resources
> to be released to verbs. Access Layer could solve it in a very complex way
> to speed up the release, but we are not sure if all corner cases are
covered
> and it seems messy to do it all in AL.

Would the problem still exist if the consumer called ib_cm_dreq() to
disconnect
the connections, then released the resources and then called ib_close_al() ?

In other words, is the problem a deadlock problem because the consumer has
called
ib_close_al() but the connections can't complete because the consumer hasn't
freed
their resources yet and so the TIMEWAIT state will never terminate ?

As you can see, I don't yet understand the problem you are trying to solve.

Thanks,

Joe

>

>

>
>
> the changes we had in mind are to push this responsibilty to the Verbs
Layer
> to acheive an implementable and managable solution.
>
> - ib_qp_mod_t change
>    currently has no parameters for modify to IB_QPS_RESET and IB_QPS_ERROR
> states. The change proposed will add another member to this modify_qp
> structure, so that Access Layer can now pass an absolute time time that
> specifies the TIMEWAIT period.
>
>  This change is required since the Verbs driver does not have knowledge
> about the round trip delay indicated by packet_lifetime in the
> path record.
>
> ci_modify_qp() changes to verb driver
> Further modify_qp calls to RTR state will check if this time has elapsed.
If
> the time period has not elapsed, then the subsequent modify_qp to RTR is
> failed with a new return code IB_QP_IN_TIMEWAIT, or IB_EE_IN_TIMEWAIT to
> indicate the effect.
>
> - ci_destroy_qp()
> Access Layer will modify the QP state to reset before calling
> ci_destroy_qp(), there by setting the timewait.
>
> Verbs drivers must gaurantee that any QP in timedwait state that are being
> destroyed are not re-allocated in a subsequent
> ci_create_qp() call.
>
> Con: If a consumer is not using the AL CM for managing connections,  and
is
> not setting the TIME_WAIT value during a reset/error transition, its
> possible that a QP may be reallocated and reused before the timewait has
> expired. This would mean "architecturally" a stale packet may be
delivered.
> Also a slight deviation from IB spec that has no parameters for modify_qp
to
> reset/error state.
>
> Pro: This solution makes channel driver/application shutdown handled well
> without too much work in Access Layer trying to handle this condition.
>
> --
> Ashok Raj                                          Phone :(503)-712-5966
> Advanced Chipset Division                   Fax     :(503)-712-1429
> Platform Products Group
> Mail Stop: JF5-361
> 5200, NE Elam Young Parkway
> Hillsboro, OR - 97124
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Infiniband-general mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-general



-------------------------------------------------------
This sf.net email is sponsored by: Dice - The leading online job board
for high-tech professionals. Search and apply for tech jobs today!
http://seeker.dice.com/seeker.epl?rel_code=31
_______________________________________________
Infiniband-general mailing list
Inf...@li...
https://lists.sourceforge.net/lists/listinfo/infiniband-general

"Raj, Ashok" wrote:

Thanks very much.

Joe

> OpenSM will support only one version (i.e 1.1). I had received from interim
> drafts earlier,
> but not sure if they have changed a lot since then. The spec release date
> was Sept last i heard.
>
> -----Original Message-----
> From: Joseph Golio [mailto:go...@vi...]
> Sent: Monday, July 22, 2002 5:31 AM
> To: Larsen, Roy K
> Cc: 'Rosenstock, Hal'; Lomartire, David M; InfiniBand Access Layer;
> InfiniBand Early_Reference; InfiniBand General; InfiniBand HCA_DDK;
> InfiniBand IPoIB
> Subject: [Infiniband-hca_ddk] Re: [Infiniband-ipoib] RE:
> [Infiniband-general] New InfiniBand code drop available
>
> "Larsen, Roy K" wrote:
>
> All,
>
>     FYI -- I asked the IETF (on the reflector) which version of the IBTA
> spec
> (as far as multicast)
> they are targeting and the response was v1.1 (as you would expect).
>
> Out of curiosity, which version of the spec does the OpenSM support. Both,
> 1.0
> or 1.1 ?
>
> Thanks,
>
> Joe
>
> > Hal,
> >
> > The IPoIB driver uses AL interfaces for interactions with the SM.  Since
> > OpenSM is part of the project, it's a safe bet that it will interoperate.
> > However, the InfinaBand project is not far enough along to have done any
> > integration testing.  (The IPoIB code posted uses it's own AL stub for
> unit
> > testing.)  With respect to other SM implementations, AL uses MAD frames to
> > get, set, and query the SM.  So, to the extent that the SM in questions
> > supports the needed operations, I don't see any reason why the project
> > shouldn't interoperate.  It certainly is the goal and the project would be
> > happy to work with other SM implementations to resolve any compliance
> > issues.
> >
> > The IPoIB IETF working group is still wading through the multicast issues.
> > As it stands now, it will depend on multicast services that have yet to be
> > published by the IBTA (but will be shortly).  The IPoIB implementation
> that
> > was just posted does not yet have code to cover general multicast - just
> an
> > explicit join of the broadcast/all-nodes multicast group.
> >
> > To follow up on that last sentence.  The implementation that was posted is
> > not complete.  As stated above, the general multicast code is missing as
> > well as the send and shadow ARP sections.  It is also missing the user and
> > kernel mode interfaces to resolve an IP address to an address vector
> (needed
> > by SDP).  However, the CA configuration discovery and initialization code
> is
> > complete and unit tested (with a stub AL included in the IPoIB project)
> > which provides example interfaces to AL and the component library.
> >
> > Roy
> >
> > -----Original Message-----
> > From: Rosenstock, Hal [mailto:ha...@pa...]
> > Sent: Wednesday, July 17, 2002 9:38 AM
> > To: Lomartire, David M
> > Cc: InfiniBand Access Layer; InfiniBand Early_Reference; InfiniBand
> > General; InfiniBand HCA_DDK; InfiniBand IPoIB
> > Subject: [Infiniband-ipoib] RE: [Infiniband-general] New InfiniBand code
> > drop available
> >
> > Hi,
> >
> > Does the IPoIB work with a particular SM ? Does it work with OpenSM ? Does
> > it work with the Brazos River edition of Lane 15 ?
> >
> > Also, what version of IB multicast is used ? Is this 1.0.a or some pre 1.1
> > IB multicast ? Perhaps this is an interface layer question.
> >
> > Thanks.
> >
> > -- Hal
> >
> > -----Original Message-----
> > From: Lomartire, David M [mailto:dav...@in...]
> > Sent: Tuesday, July 16, 2002 7:11 PM
> > To: InfiniBand Access Layer; InfiniBand Early_Reference; InfiniBand
> > General; InfiniBand HCA_DDK; InfiniBand IPoIB; InfiniBand SDP;
> > InfiniBand SRP
> > Subject: [Infiniband-general] New InfiniBand code drop available
> >
> > The iba_020716 code drop has just been posted to the Linux InfiniBand
> > Project.  See the News (https://sourceforge.net/news/?group_id=52127) for
> > the latest details.
> >
> > In summary, this drop contains the following:
> >
> > *       Component Library (include/iba/complib and drivers/iba/complib)
> > *       Kernel verbs header file (include/iba/ib_ci.h)
> > *       Sample HCA driver scaffold (drivers/iba/hca/model)
> > *       Usermode verbs header file (include/iba/ib_uvp.h)
> > *       Access Layer (drivers/iba/al)
> > *       IPoIB (drivers/iba/net/ipoib)
> > *       Subnet Manager (linuxuser/iba/OpenSM) - this is just the start of
> > the implementation and more details will be posted soon at the "Subnet
> > Management" sub-project of the Linux InfiniBand Project home page
> > (http://infiniband.sourceforge.net/)
> > *       API documentation (iba_docs) - this is the same documentation that
> > is posted on the home page at the "Documents" sections of the "HCA DDK"
> and
> > "Access Layer" sub-projects
> >
> > Important note: This code is not yet in the project CVS repository!  Once
> > CVS has been updated, an announcement will be sent out.  So, for now, the
> > only place to get this latest code is from the tarball posted at the file
> > list at http://sourceforge.net/project/showfiles.php?group_id=52127.
> >
> >    *  Dave
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by: Jabber - The world's fastest growing
> > real-time communications platform! Don't just IM. Build it in!
> > http://www.jabber.com/osdn/xim
> > _______________________________________________
> > Infiniband-general mailing list
> > Inf...@li...
> > https://lists.sourceforge.net/lists/listinfo/infiniband-general
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Infiniband-ipoib mailing list
> > Inf...@li...
> > https://lists.sourceforge.net/lists/listinfo/infiniband-ipoib
> >
> > -------------------------------------------------------
> > This sf.net email is sponsored by:ThinkGeek
> > Welcome to geek heaven.
> > http://thinkgeek.com/sf
> > _______________________________________________
> > Infiniband-ipoib mailing list
> > Inf...@li...
> > https://lists.sourceforge.net/lists/listinfo/infiniband-ipoib
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Infiniband-hca_ddk mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-hca_ddk

OpenSM will support only one version (i.e 1.1). I had received from interim
drafts earlier, 
but not sure if they have changed a lot since then. The spec release date
was Sept last i heard.

-----Original Message-----
From: Joseph Golio [mailto:go...@vi...]
Sent: Monday, July 22, 2002 5:31 AM
To: Larsen, Roy K
Cc: 'Rosenstock, Hal'; Lomartire, David M; InfiniBand Access Layer;
InfiniBand Early_Reference; InfiniBand General; InfiniBand HCA_DDK;
InfiniBand IPoIB
Subject: [Infiniband-hca_ddk] Re: [Infiniband-ipoib] RE:
[Infiniband-general] New InfiniBand code drop available


"Larsen, Roy K" wrote:

All,

    FYI -- I asked the IETF (on the reflector) which version of the IBTA
spec
(as far as multicast)
they are targeting and the response was v1.1 (as you would expect).

Out of curiosity, which version of the spec does the OpenSM support. Both,
1.0
or 1.1 ?

Thanks,

Joe

> Hal,
>
> The IPoIB driver uses AL interfaces for interactions with the SM.  Since
> OpenSM is part of the project, it's a safe bet that it will interoperate.
> However, the InfinaBand project is not far enough along to have done any
> integration testing.  (The IPoIB code posted uses it's own AL stub for
unit
> testing.)  With respect to other SM implementations, AL uses MAD frames to
> get, set, and query the SM.  So, to the extent that the SM in questions
> supports the needed operations, I don't see any reason why the project
> shouldn't interoperate.  It certainly is the goal and the project would be
> happy to work with other SM implementations to resolve any compliance
> issues.
>
> The IPoIB IETF working group is still wading through the multicast issues.
> As it stands now, it will depend on multicast services that have yet to be
> published by the IBTA (but will be shortly).  The IPoIB implementation
that
> was just posted does not yet have code to cover general multicast - just
an
> explicit join of the broadcast/all-nodes multicast group.
>
> To follow up on that last sentence.  The implementation that was posted is
> not complete.  As stated above, the general multicast code is missing as
> well as the send and shadow ARP sections.  It is also missing the user and
> kernel mode interfaces to resolve an IP address to an address vector
(needed
> by SDP).  However, the CA configuration discovery and initialization code
is
> complete and unit tested (with a stub AL included in the IPoIB project)
> which provides example interfaces to AL and the component library.
>
> Roy
>
> -----Original Message-----
> From: Rosenstock, Hal [mailto:ha...@pa...]
> Sent: Wednesday, July 17, 2002 9:38 AM
> To: Lomartire, David M
> Cc: InfiniBand Access Layer; InfiniBand Early_Reference; InfiniBand
> General; InfiniBand HCA_DDK; InfiniBand IPoIB
> Subject: [Infiniband-ipoib] RE: [Infiniband-general] New InfiniBand code
> drop available
>
> Hi,
>
> Does the IPoIB work with a particular SM ? Does it work with OpenSM ? Does
> it work with the Brazos River edition of Lane 15 ?
>
> Also, what version of IB multicast is used ? Is this 1.0.a or some pre 1.1
> IB multicast ? Perhaps this is an interface layer question.
>
> Thanks.
>
> -- Hal
>
> -----Original Message-----
> From: Lomartire, David M [mailto:dav...@in...]
> Sent: Tuesday, July 16, 2002 7:11 PM
> To: InfiniBand Access Layer; InfiniBand Early_Reference; InfiniBand
> General; InfiniBand HCA_DDK; InfiniBand IPoIB; InfiniBand SDP;
> InfiniBand SRP
> Subject: [Infiniband-general] New InfiniBand code drop available
>
> The iba_020716 code drop has just been posted to the Linux InfiniBand
> Project.  See the News (https://sourceforge.net/news/?group_id=52127) for
> the latest details.
>
> In summary, this drop contains the following:
>
> *       Component Library (include/iba/complib and drivers/iba/complib)
> *       Kernel verbs header file (include/iba/ib_ci.h)
> *       Sample HCA driver scaffold (drivers/iba/hca/model)
> *       Usermode verbs header file (include/iba/ib_uvp.h)
> *       Access Layer (drivers/iba/al)
> *       IPoIB (drivers/iba/net/ipoib)
> *       Subnet Manager (linuxuser/iba/OpenSM) - this is just the start of
> the implementation and more details will be posted soon at the "Subnet
> Management" sub-project of the Linux InfiniBand Project home page
> (http://infiniband.sourceforge.net/)
> *       API documentation (iba_docs) - this is the same documentation that
> is posted on the home page at the "Documents" sections of the "HCA DDK"
and
> "Access Layer" sub-projects
>
> Important note: This code is not yet in the project CVS repository!  Once
> CVS has been updated, an announcement will be sent out.  So, for now, the
> only place to get this latest code is from the tarball posted at the file
> list at http://sourceforge.net/project/showfiles.php?group_id=52127.
>
>    *  Dave
>
> -------------------------------------------------------
> This sf.net email is sponsored by: Jabber - The world's fastest growing
> real-time communications platform! Don't just IM. Build it in!
> http://www.jabber.com/osdn/xim
> _______________________________________________
> Infiniband-general mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-general
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Infiniband-ipoib mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-ipoib
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Infiniband-ipoib mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-ipoib



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Infiniband-hca_ddk mailing list
Inf...@li...
https://lists.sourceforge.net/lists/listinfo/infiniband-hca_ddk

"Larsen, Roy K" wrote:

All,

    FYI -- I asked the IETF (on the reflector) which version of the IBTA spec
(as far as multicast)
they are targeting and the response was v1.1 (as you would expect).

Out of curiosity, which version of the spec does the OpenSM support. Both, 1.0
or 1.1 ?

Thanks,

Joe

> Hal,
>
> The IPoIB driver uses AL interfaces for interactions with the SM.  Since
> OpenSM is part of the project, it's a safe bet that it will interoperate.
> However, the InfinaBand project is not far enough along to have done any
> integration testing.  (The IPoIB code posted uses it's own AL stub for unit
> testing.)  With respect to other SM implementations, AL uses MAD frames to
> get, set, and query the SM.  So, to the extent that the SM in questions
> supports the needed operations, I don't see any reason why the project
> shouldn't interoperate.  It certainly is the goal and the project would be
> happy to work with other SM implementations to resolve any compliance
> issues.
>
> The IPoIB IETF working group is still wading through the multicast issues.
> As it stands now, it will depend on multicast services that have yet to be
> published by the IBTA (but will be shortly).  The IPoIB implementation that
> was just posted does not yet have code to cover general multicast - just an
> explicit join of the broadcast/all-nodes multicast group.
>
> To follow up on that last sentence.  The implementation that was posted is
> not complete.  As stated above, the general multicast code is missing as
> well as the send and shadow ARP sections.  It is also missing the user and
> kernel mode interfaces to resolve an IP address to an address vector (needed
> by SDP).  However, the CA configuration discovery and initialization code is
> complete and unit tested (with a stub AL included in the IPoIB project)
> which provides example interfaces to AL and the component library.
>
> Roy
>
> -----Original Message-----
> From: Rosenstock, Hal [mailto:ha...@pa...]
> Sent: Wednesday, July 17, 2002 9:38 AM
> To: Lomartire, David M
> Cc: InfiniBand Access Layer; InfiniBand Early_Reference; InfiniBand
> General; InfiniBand HCA_DDK; InfiniBand IPoIB
> Subject: [Infiniband-ipoib] RE: [Infiniband-general] New InfiniBand code
> drop available
>
> Hi,
>
> Does the IPoIB work with a particular SM ? Does it work with OpenSM ? Does
> it work with the Brazos River edition of Lane 15 ?
>
> Also, what version of IB multicast is used ? Is this 1.0.a or some pre 1.1
> IB multicast ? Perhaps this is an interface layer question.
>
> Thanks.
>
> -- Hal
>
> -----Original Message-----
> From: Lomartire, David M [mailto:dav...@in...]
> Sent: Tuesday, July 16, 2002 7:11 PM
> To: InfiniBand Access Layer; InfiniBand Early_Reference; InfiniBand
> General; InfiniBand HCA_DDK; InfiniBand IPoIB; InfiniBand SDP;
> InfiniBand SRP
> Subject: [Infiniband-general] New InfiniBand code drop available
>
> The iba_020716 code drop has just been posted to the Linux InfiniBand
> Project.  See the News (https://sourceforge.net/news/?group_id=52127) for
> the latest details.
>
> In summary, this drop contains the following:
>
> *       Component Library (include/iba/complib and drivers/iba/complib)
> *       Kernel verbs header file (include/iba/ib_ci.h)
> *       Sample HCA driver scaffold (drivers/iba/hca/model)
> *       Usermode verbs header file (include/iba/ib_uvp.h)
> *       Access Layer (drivers/iba/al)
> *       IPoIB (drivers/iba/net/ipoib)
> *       Subnet Manager (linuxuser/iba/OpenSM) - this is just the start of
> the implementation and more details will be posted soon at the "Subnet
> Management" sub-project of the Linux InfiniBand Project home page
> (http://infiniband.sourceforge.net/)
> *       API documentation (iba_docs) - this is the same documentation that
> is posted on the home page at the "Documents" sections of the "HCA DDK" and
> "Access Layer" sub-projects
>
> Important note: This code is not yet in the project CVS repository!  Once
> CVS has been updated, an announcement will be sent out.  So, for now, the
> only place to get this latest code is from the tarball posted at the file
> list at http://sourceforge.net/project/showfiles.php?group_id=52127.
>
>    *  Dave
>
> -------------------------------------------------------
> This sf.net email is sponsored by: Jabber - The world's fastest growing
> real-time communications platform! Don't just IM. Build it in!
> http://www.jabber.com/osdn/xim
> _______________________________________________
> Infiniband-general mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-general
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Infiniband-ipoib mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-ipoib
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Infiniband-ipoib mailing list
> Inf...@li...
> https://lists.sourceforge.net/lists/listinfo/infiniband-ipoib