2009-07-14 06:34:54 +08:00
|
|
|
/*
|
|
|
|
* Copyright (c) 2009, Microsoft Corporation.
|
|
|
|
*
|
|
|
|
* This program is free software; you can redistribute it and/or modify it
|
|
|
|
* under the terms and conditions of the GNU General Public License,
|
|
|
|
* version 2, as published by the Free Software Foundation.
|
|
|
|
*
|
|
|
|
* This program is distributed in the hope it will be useful, but WITHOUT
|
|
|
|
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
|
|
|
* FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
|
|
|
|
* more details.
|
|
|
|
*
|
|
|
|
* You should have received a copy of the GNU General Public License along with
|
2013-12-06 22:28:47 +08:00
|
|
|
* this program; if not, see <http://www.gnu.org/licenses/>.
|
2009-07-14 06:34:54 +08:00
|
|
|
*
|
|
|
|
* Authors:
|
2009-11-24 01:00:22 +08:00
|
|
|
* Haiyang Zhang <haiyangz@microsoft.com>
|
2009-07-14 06:34:54 +08:00
|
|
|
* Hank Janssen <hjanssen@microsoft.com>
|
|
|
|
*/
|
2011-03-30 04:58:48 +08:00
|
|
|
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
|
|
|
|
|
2009-07-15 06:08:20 +08:00
|
|
|
#include <linux/kernel.h>
|
2011-02-12 01:59:43 +08:00
|
|
|
#include <linux/sched.h>
|
|
|
|
#include <linux/wait.h>
|
2009-07-16 02:06:01 +08:00
|
|
|
#include <linux/mm.h>
|
2009-07-17 02:50:41 +08:00
|
|
|
#include <linux/delay.h>
|
2009-09-03 01:33:05 +08:00
|
|
|
#include <linux/io.h>
|
include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h
percpu.h is included by sched.h and module.h and thus ends up being
included when building most .c files. percpu.h includes slab.h which
in turn includes gfp.h making everything defined by the two files
universally available and complicating inclusion dependencies.
percpu.h -> slab.h dependency is about to be removed. Prepare for
this change by updating users of gfp and slab facilities include those
headers directly instead of assuming availability. As this conversion
needs to touch large number of source files, the following script is
used as the basis of conversion.
http://userweb.kernel.org/~tj/misc/slabh-sweep.py
The script does the followings.
* Scan files for gfp and slab usages and update includes such that
only the necessary includes are there. ie. if only gfp is used,
gfp.h, if slab is used, slab.h.
* When the script inserts a new include, it looks at the include
blocks and try to put the new include such that its order conforms
to its surrounding. It's put in the include block which contains
core kernel includes, in the same order that the rest are ordered -
alphabetical, Christmas tree, rev-Xmas-tree or at the end if there
doesn't seem to be any matching order.
* If the script can't find a place to put a new include (mostly
because the file doesn't have fitting include block), it prints out
an error message indicating which .h file needs to be added to the
file.
The conversion was done in the following steps.
1. The initial automatic conversion of all .c files updated slightly
over 4000 files, deleting around 700 includes and adding ~480 gfp.h
and ~3000 slab.h inclusions. The script emitted errors for ~400
files.
2. Each error was manually checked. Some didn't need the inclusion,
some needed manual addition while adding it to implementation .h or
embedding .c file was more appropriate for others. This step added
inclusions to around 150 files.
3. The script was run again and the output was compared to the edits
from #2 to make sure no file was left behind.
4. Several build tests were done and a couple of problems were fixed.
e.g. lib/decompress_*.c used malloc/free() wrappers around slab
APIs requiring slab.h to be added manually.
5. The script was run on all .h files but without automatically
editing them as sprinkling gfp.h and slab.h inclusions around .h
files could easily lead to inclusion dependency hell. Most gfp.h
inclusion directives were ignored as stuff from gfp.h was usually
wildly available and often used in preprocessor macros. Each
slab.h inclusion directive was examined and added manually as
necessary.
6. percpu.h was updated not to include slab.h.
7. Build test were done on the following configurations and failures
were fixed. CONFIG_GCOV_KERNEL was turned off for all tests (as my
distributed build env didn't work with gcov compiles) and a few
more options had to be turned off depending on archs to make things
build (like ipr on powerpc/64 which failed due to missing writeq).
* x86 and x86_64 UP and SMP allmodconfig and a custom test config.
* powerpc and powerpc64 SMP allmodconfig
* sparc and sparc64 SMP allmodconfig
* ia64 SMP allmodconfig
* s390 SMP allmodconfig
* alpha SMP allmodconfig
* um on x86_64 SMP allmodconfig
8. percpu.h modifications were reverted so that it could be applied as
a separate patch and serve as bisection point.
Given the fact that I had only a couple of failures from tests on step
6, I'm fairly confident about the coverage of this conversion patch.
If there is a breakage, it's likely to be something in one of the arch
headers which should be easily discoverable easily on most builds of
the specific arch.
Signed-off-by: Tejun Heo <tj@kernel.org>
Guess-its-ok-by: Christoph Lameter <cl@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Lee Schermerhorn <Lee.Schermerhorn@hp.com>
2010-03-24 16:04:11 +08:00
|
|
|
#include <linux/slab.h>
|
2011-09-02 03:19:41 +08:00
|
|
|
#include <linux/netdevice.h>
|
2011-12-16 05:45:16 +08:00
|
|
|
#include <linux/if_ether.h>
|
2015-06-02 17:01:38 +08:00
|
|
|
#include <linux/vmalloc.h>
|
2014-05-01 01:14:31 +08:00
|
|
|
#include <asm/sync_bitops.h>
|
2011-05-13 10:34:15 +08:00
|
|
|
|
2011-05-13 10:34:37 +08:00
|
|
|
#include "hyperv_net.h"
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2016-04-15 07:31:54 +08:00
|
|
|
/*
|
|
|
|
* Switch the data path from the synthetic interface to the VF
|
|
|
|
* interface.
|
|
|
|
*/
|
2016-05-13 19:55:23 +08:00
|
|
|
void netvsc_switch_datapath(struct net_device *ndev, bool vf)
|
2016-04-15 07:31:54 +08:00
|
|
|
{
|
2016-05-13 19:55:22 +08:00
|
|
|
struct net_device_context *net_device_ctx = netdev_priv(ndev);
|
|
|
|
struct hv_device *dev = net_device_ctx->device_ctx;
|
2016-05-13 19:55:23 +08:00
|
|
|
struct netvsc_device *nv_dev = net_device_ctx->nvdev;
|
|
|
|
struct nvsp_message *init_pkt = &nv_dev->channel_init_pkt;
|
2016-04-15 07:31:54 +08:00
|
|
|
|
|
|
|
memset(init_pkt, 0, sizeof(struct nvsp_message));
|
|
|
|
init_pkt->hdr.msg_type = NVSP_MSG4_TYPE_SWITCH_DATA_PATH;
|
|
|
|
if (vf)
|
|
|
|
init_pkt->msg.v4_msg.active_dp.active_datapath =
|
|
|
|
NVSP_DATAPATH_VF;
|
|
|
|
else
|
|
|
|
init_pkt->msg.v4_msg.active_dp.active_datapath =
|
|
|
|
NVSP_DATAPATH_SYNTHETIC;
|
|
|
|
|
|
|
|
vmbus_sendpacket(dev->channel, init_pkt,
|
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
(unsigned long)init_pkt,
|
|
|
|
VM_PKT_DATA_INBAND, 0);
|
|
|
|
}
|
|
|
|
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2016-05-13 19:55:25 +08:00
|
|
|
static struct netvsc_device *alloc_net_device(void)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2010-12-11 04:03:54 +08:00
|
|
|
struct netvsc_device *net_device;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
net_device = kzalloc(sizeof(struct netvsc_device), GFP_KERNEL);
|
|
|
|
if (!net_device)
|
2009-07-14 06:34:54 +08:00
|
|
|
return NULL;
|
|
|
|
|
2014-08-16 03:18:19 +08:00
|
|
|
net_device->cb_buffer = kzalloc(NETVSC_PACKET_SIZE, GFP_KERNEL);
|
|
|
|
if (!net_device->cb_buffer) {
|
|
|
|
kfree(net_device);
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2012-06-04 14:42:38 +08:00
|
|
|
init_waitqueue_head(&net_device->wait_drain);
|
2011-08-28 02:31:12 +08:00
|
|
|
net_device->destroy = false;
|
2016-04-15 07:31:54 +08:00
|
|
|
atomic_set(&net_device->open_cnt, 0);
|
|
|
|
atomic_set(&net_device->vf_use_cnt, 0);
|
2015-03-27 00:03:37 +08:00
|
|
|
net_device->max_pkt = RNDIS_MAX_PKT_DEFAULT;
|
|
|
|
net_device->pkt_align = RNDIS_PKT_ALIGN_DEFAULT;
|
|
|
|
|
2016-04-15 07:31:54 +08:00
|
|
|
net_device->vf_netdev = NULL;
|
|
|
|
net_device->vf_inject = false;
|
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
return net_device;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2014-08-16 03:18:19 +08:00
|
|
|
static void free_netvsc_device(struct netvsc_device *nvdev)
|
|
|
|
{
|
|
|
|
kfree(nvdev->cb_buffer);
|
|
|
|
kfree(nvdev);
|
|
|
|
}
|
|
|
|
|
2010-12-11 04:03:55 +08:00
|
|
|
static struct netvsc_device *get_outbound_net_device(struct hv_device *device)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2016-06-03 23:50:59 +08:00
|
|
|
struct netvsc_device *net_device = hv_device_to_netvsc_device(device);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2011-08-28 02:31:16 +08:00
|
|
|
if (net_device && net_device->destroy)
|
2010-12-11 04:03:54 +08:00
|
|
|
net_device = NULL;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
return net_device;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2010-12-11 04:03:55 +08:00
|
|
|
static struct netvsc_device *get_inbound_net_device(struct hv_device *device)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2016-06-03 23:50:59 +08:00
|
|
|
struct netvsc_device *net_device = hv_device_to_netvsc_device(device);
|
2011-08-28 02:31:16 +08:00
|
|
|
|
|
|
|
if (!net_device)
|
|
|
|
goto get_in_err;
|
|
|
|
|
|
|
|
if (net_device->destroy &&
|
|
|
|
atomic_read(&net_device->num_outstanding_sends) == 0)
|
2010-12-11 04:03:54 +08:00
|
|
|
net_device = NULL;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2011-08-28 02:31:16 +08:00
|
|
|
get_in_err:
|
2010-12-11 04:03:54 +08:00
|
|
|
return net_device;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
2016-05-13 19:55:22 +08:00
|
|
|
static int netvsc_destroy_buf(struct hv_device *device)
|
2011-04-22 03:30:43 +08:00
|
|
|
{
|
|
|
|
struct nvsp_message *revoke_packet;
|
|
|
|
int ret = 0;
|
2016-05-13 19:55:22 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
2016-06-03 23:50:59 +08:00
|
|
|
struct netvsc_device *net_device = net_device_to_netvsc_device(ndev);
|
2011-04-22 03:30:43 +08:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If we got a section count, it means we received a
|
|
|
|
* SendReceiveBufferComplete msg (ie sent
|
|
|
|
* NvspMessage1TypeSendReceiveBuffer msg) therefore, we need
|
|
|
|
* to send a revoke msg here
|
|
|
|
*/
|
|
|
|
if (net_device->recv_section_cnt) {
|
|
|
|
/* Send the revoke receive buffer */
|
|
|
|
revoke_packet = &net_device->revoke_packet;
|
|
|
|
memset(revoke_packet, 0, sizeof(struct nvsp_message));
|
|
|
|
|
|
|
|
revoke_packet->hdr.msg_type =
|
|
|
|
NVSP_MSG1_TYPE_REVOKE_RECV_BUF;
|
|
|
|
revoke_packet->msg.v1_msg.
|
|
|
|
revoke_recv_buf.id = NETVSC_RECEIVE_BUFFER_ID;
|
|
|
|
|
2016-05-13 19:55:22 +08:00
|
|
|
ret = vmbus_sendpacket(device->channel,
|
2011-04-22 03:30:43 +08:00
|
|
|
revoke_packet,
|
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
(unsigned long)revoke_packet,
|
|
|
|
VM_PKT_DATA_INBAND, 0);
|
|
|
|
/*
|
|
|
|
* If we failed here, we might as well return and
|
|
|
|
* have a leak rather than continue and a bugchk
|
|
|
|
*/
|
|
|
|
if (ret != 0) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "unable to send "
|
2011-09-02 03:19:40 +08:00
|
|
|
"revoke receive buffer to netvsp\n");
|
2011-08-26 00:49:12 +08:00
|
|
|
return ret;
|
2011-04-22 03:30:43 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Teardown the gpadl on the vsp end */
|
|
|
|
if (net_device->recv_buf_gpadl_handle) {
|
2016-05-13 19:55:22 +08:00
|
|
|
ret = vmbus_teardown_gpadl(device->channel,
|
|
|
|
net_device->recv_buf_gpadl_handle);
|
2011-04-22 03:30:43 +08:00
|
|
|
|
|
|
|
/* If we failed here, we might as well return and have a leak
|
|
|
|
* rather than continue and a bugchk
|
|
|
|
*/
|
|
|
|
if (ret != 0) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev,
|
2011-09-02 03:19:40 +08:00
|
|
|
"unable to teardown receive buffer's gpadl\n");
|
2011-08-27 19:06:07 +08:00
|
|
|
return ret;
|
2011-04-22 03:30:43 +08:00
|
|
|
}
|
|
|
|
net_device->recv_buf_gpadl_handle = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (net_device->recv_buf) {
|
|
|
|
/* Free up the receive buffer */
|
2014-01-28 07:03:42 +08:00
|
|
|
vfree(net_device->recv_buf);
|
2011-04-22 03:30:43 +08:00
|
|
|
net_device->recv_buf = NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (net_device->recv_section) {
|
|
|
|
net_device->recv_section_cnt = 0;
|
|
|
|
kfree(net_device->recv_section);
|
|
|
|
net_device->recv_section = NULL;
|
|
|
|
}
|
|
|
|
|
2014-05-01 01:14:31 +08:00
|
|
|
/* Deal with the send buffer we may have setup.
|
|
|
|
* If we got a send section size, it means we received a
|
2014-12-20 10:25:18 +08:00
|
|
|
* NVSP_MSG1_TYPE_SEND_SEND_BUF_COMPLETE msg (ie sent
|
|
|
|
* NVSP_MSG1_TYPE_SEND_SEND_BUF msg) therefore, we need
|
2014-05-01 01:14:31 +08:00
|
|
|
* to send a revoke msg here
|
|
|
|
*/
|
|
|
|
if (net_device->send_section_size) {
|
|
|
|
/* Send the revoke receive buffer */
|
|
|
|
revoke_packet = &net_device->revoke_packet;
|
|
|
|
memset(revoke_packet, 0, sizeof(struct nvsp_message));
|
|
|
|
|
|
|
|
revoke_packet->hdr.msg_type =
|
|
|
|
NVSP_MSG1_TYPE_REVOKE_SEND_BUF;
|
2014-12-20 10:25:18 +08:00
|
|
|
revoke_packet->msg.v1_msg.revoke_send_buf.id =
|
|
|
|
NETVSC_SEND_BUFFER_ID;
|
2014-05-01 01:14:31 +08:00
|
|
|
|
2016-05-13 19:55:22 +08:00
|
|
|
ret = vmbus_sendpacket(device->channel,
|
2014-05-01 01:14:31 +08:00
|
|
|
revoke_packet,
|
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
(unsigned long)revoke_packet,
|
|
|
|
VM_PKT_DATA_INBAND, 0);
|
|
|
|
/* If we failed here, we might as well return and
|
|
|
|
* have a leak rather than continue and a bugchk
|
|
|
|
*/
|
|
|
|
if (ret != 0) {
|
|
|
|
netdev_err(ndev, "unable to send "
|
|
|
|
"revoke send buffer to netvsp\n");
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
/* Teardown the gpadl on the vsp end */
|
|
|
|
if (net_device->send_buf_gpadl_handle) {
|
2016-05-13 19:55:22 +08:00
|
|
|
ret = vmbus_teardown_gpadl(device->channel,
|
2014-05-01 01:14:31 +08:00
|
|
|
net_device->send_buf_gpadl_handle);
|
|
|
|
|
|
|
|
/* If we failed here, we might as well return and have a leak
|
|
|
|
* rather than continue and a bugchk
|
|
|
|
*/
|
|
|
|
if (ret != 0) {
|
|
|
|
netdev_err(ndev,
|
|
|
|
"unable to teardown send buffer's gpadl\n");
|
|
|
|
return ret;
|
|
|
|
}
|
2014-06-17 04:59:02 +08:00
|
|
|
net_device->send_buf_gpadl_handle = 0;
|
2014-05-01 01:14:31 +08:00
|
|
|
}
|
|
|
|
if (net_device->send_buf) {
|
2014-12-20 10:25:18 +08:00
|
|
|
/* Free up the send buffer */
|
2014-08-03 01:42:02 +08:00
|
|
|
vfree(net_device->send_buf);
|
2014-05-01 01:14:31 +08:00
|
|
|
net_device->send_buf = NULL;
|
|
|
|
}
|
|
|
|
kfree(net_device->send_section_map);
|
|
|
|
|
2011-04-22 03:30:43 +08:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2014-05-01 01:14:31 +08:00
|
|
|
static int netvsc_init_buf(struct hv_device *device)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2009-09-03 01:33:05 +08:00
|
|
|
int ret = 0;
|
2010-12-11 04:03:54 +08:00
|
|
|
struct netvsc_device *net_device;
|
|
|
|
struct nvsp_message *init_packet;
|
2011-09-14 01:59:49 +08:00
|
|
|
struct net_device *ndev;
|
2015-05-29 08:08:06 +08:00
|
|
|
int node;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:55 +08:00
|
|
|
net_device = get_outbound_net_device(device);
|
2011-09-14 01:59:49 +08:00
|
|
|
if (!net_device)
|
2011-08-26 00:49:13 +08:00
|
|
|
return -ENODEV;
|
2016-05-13 19:55:23 +08:00
|
|
|
ndev = hv_get_drvdata(device);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2015-05-29 08:08:06 +08:00
|
|
|
node = cpu_to_node(device->channel->target_cpu);
|
|
|
|
net_device->recv_buf = vzalloc_node(net_device->recv_buf_size, node);
|
|
|
|
if (!net_device->recv_buf)
|
|
|
|
net_device->recv_buf = vzalloc(net_device->recv_buf_size);
|
|
|
|
|
2010-12-11 04:03:59 +08:00
|
|
|
if (!net_device->recv_buf) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "unable to allocate receive "
|
2011-09-02 03:19:40 +08:00
|
|
|
"buffer of size %d\n", net_device->recv_buf_size);
|
2011-08-26 00:49:13 +08:00
|
|
|
ret = -ENOMEM;
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/*
|
|
|
|
* Establish the gpadl handle for this buffer on this
|
|
|
|
* channel. Note: This call uses the vmbus connection rather
|
|
|
|
* than the channel to establish the gpadl handle.
|
|
|
|
*/
|
2010-12-11 04:03:59 +08:00
|
|
|
ret = vmbus_establish_gpadl(device->channel, net_device->recv_buf,
|
|
|
|
net_device->recv_buf_size,
|
|
|
|
&net_device->recv_buf_gpadl_handle);
|
2009-09-03 01:33:05 +08:00
|
|
|
if (ret != 0) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev,
|
2011-09-02 03:19:40 +08:00
|
|
|
"unable to establish receive buffer's gpadl\n");
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Notify the NetVsp of the gpadl handle */
|
2010-12-11 04:03:59 +08:00
|
|
|
init_packet = &net_device->channel_init_pkt;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
memset(init_packet, 0, sizeof(struct nvsp_message));
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:59 +08:00
|
|
|
init_packet->hdr.msg_type = NVSP_MSG1_TYPE_SEND_RECV_BUF;
|
|
|
|
init_packet->msg.v1_msg.send_recv_buf.
|
|
|
|
gpadl_handle = net_device->recv_buf_gpadl_handle;
|
|
|
|
init_packet->msg.v1_msg.
|
|
|
|
send_recv_buf.id = NETVSC_RECEIVE_BUFFER_ID;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Send the gpadl notification request */
|
2010-12-11 04:03:54 +08:00
|
|
|
ret = vmbus_sendpacket(device->channel, init_packet,
|
2010-10-22 00:43:24 +08:00
|
|
|
sizeof(struct nvsp_message),
|
2010-12-11 04:03:54 +08:00
|
|
|
(unsigned long)init_packet,
|
2011-01-27 04:12:13 +08:00
|
|
|
VM_PKT_DATA_INBAND,
|
2010-10-22 00:43:24 +08:00
|
|
|
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
|
2009-09-03 01:33:05 +08:00
|
|
|
if (ret != 0) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev,
|
2011-09-02 03:19:40 +08:00
|
|
|
"unable to send receive buffer's gpadl to netvsp\n");
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2016-06-09 18:44:03 +08:00
|
|
|
wait_for_completion(&net_device->channel_init_wait);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Check the response */
|
2010-12-11 04:03:59 +08:00
|
|
|
if (init_packet->msg.v1_msg.
|
|
|
|
send_recv_buf_complete.status != NVSP_STAT_SUCCESS) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "Unable to complete receive buffer "
|
2011-09-02 03:19:48 +08:00
|
|
|
"initialization with NetVsp - status %d\n",
|
2010-12-11 04:03:59 +08:00
|
|
|
init_packet->msg.v1_msg.
|
|
|
|
send_recv_buf_complete.status);
|
2011-08-26 00:49:13 +08:00
|
|
|
ret = -EINVAL;
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Parse the response */
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:59 +08:00
|
|
|
net_device->recv_section_cnt = init_packet->msg.
|
|
|
|
v1_msg.send_recv_buf_complete.num_sections;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2011-11-30 23:19:07 +08:00
|
|
|
net_device->recv_section = kmemdup(
|
|
|
|
init_packet->msg.v1_msg.send_recv_buf_complete.sections,
|
|
|
|
net_device->recv_section_cnt *
|
|
|
|
sizeof(struct nvsp_1_receive_buffer_section),
|
|
|
|
GFP_KERNEL);
|
2010-12-11 04:03:59 +08:00
|
|
|
if (net_device->recv_section == NULL) {
|
2011-08-26 00:49:13 +08:00
|
|
|
ret = -EINVAL;
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2009-09-03 01:33:05 +08:00
|
|
|
/*
|
|
|
|
* For 1st release, there should only be 1 section that represents the
|
|
|
|
* entire receive buffer
|
|
|
|
*/
|
2010-12-11 04:03:59 +08:00
|
|
|
if (net_device->recv_section_cnt != 1 ||
|
|
|
|
net_device->recv_section->offset != 0) {
|
2011-08-26 00:49:13 +08:00
|
|
|
ret = -EINVAL;
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2014-05-01 01:14:31 +08:00
|
|
|
/* Now setup the send buffer.
|
|
|
|
*/
|
2015-05-29 08:08:07 +08:00
|
|
|
net_device->send_buf = vzalloc_node(net_device->send_buf_size, node);
|
|
|
|
if (!net_device->send_buf)
|
|
|
|
net_device->send_buf = vzalloc(net_device->send_buf_size);
|
2014-05-01 01:14:31 +08:00
|
|
|
if (!net_device->send_buf) {
|
|
|
|
netdev_err(ndev, "unable to allocate send "
|
|
|
|
"buffer of size %d\n", net_device->send_buf_size);
|
|
|
|
ret = -ENOMEM;
|
|
|
|
goto cleanup;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Establish the gpadl handle for this buffer on this
|
|
|
|
* channel. Note: This call uses the vmbus connection rather
|
|
|
|
* than the channel to establish the gpadl handle.
|
|
|
|
*/
|
|
|
|
ret = vmbus_establish_gpadl(device->channel, net_device->send_buf,
|
|
|
|
net_device->send_buf_size,
|
|
|
|
&net_device->send_buf_gpadl_handle);
|
|
|
|
if (ret != 0) {
|
|
|
|
netdev_err(ndev,
|
|
|
|
"unable to establish send buffer's gpadl\n");
|
|
|
|
goto cleanup;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Notify the NetVsp of the gpadl handle */
|
|
|
|
init_packet = &net_device->channel_init_pkt;
|
|
|
|
memset(init_packet, 0, sizeof(struct nvsp_message));
|
|
|
|
init_packet->hdr.msg_type = NVSP_MSG1_TYPE_SEND_SEND_BUF;
|
2014-12-20 10:25:18 +08:00
|
|
|
init_packet->msg.v1_msg.send_send_buf.gpadl_handle =
|
2014-05-01 01:14:31 +08:00
|
|
|
net_device->send_buf_gpadl_handle;
|
2014-12-20 10:25:18 +08:00
|
|
|
init_packet->msg.v1_msg.send_send_buf.id = NETVSC_SEND_BUFFER_ID;
|
2014-05-01 01:14:31 +08:00
|
|
|
|
|
|
|
/* Send the gpadl notification request */
|
|
|
|
ret = vmbus_sendpacket(device->channel, init_packet,
|
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
(unsigned long)init_packet,
|
|
|
|
VM_PKT_DATA_INBAND,
|
|
|
|
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
|
|
|
|
if (ret != 0) {
|
|
|
|
netdev_err(ndev,
|
|
|
|
"unable to send send buffer's gpadl to netvsp\n");
|
|
|
|
goto cleanup;
|
|
|
|
}
|
|
|
|
|
2016-06-09 18:44:03 +08:00
|
|
|
wait_for_completion(&net_device->channel_init_wait);
|
2014-05-01 01:14:31 +08:00
|
|
|
|
|
|
|
/* Check the response */
|
|
|
|
if (init_packet->msg.v1_msg.
|
|
|
|
send_send_buf_complete.status != NVSP_STAT_SUCCESS) {
|
|
|
|
netdev_err(ndev, "Unable to complete send buffer "
|
|
|
|
"initialization with NetVsp - status %d\n",
|
|
|
|
init_packet->msg.v1_msg.
|
2014-12-20 10:25:18 +08:00
|
|
|
send_send_buf_complete.status);
|
2014-05-01 01:14:31 +08:00
|
|
|
ret = -EINVAL;
|
|
|
|
goto cleanup;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Parse the response */
|
|
|
|
net_device->send_section_size = init_packet->msg.
|
|
|
|
v1_msg.send_send_buf_complete.section_size;
|
|
|
|
|
|
|
|
/* Section count is simply the size divided by the section size.
|
|
|
|
*/
|
|
|
|
net_device->send_section_cnt =
|
|
|
|
net_device->send_buf_size/net_device->send_section_size;
|
|
|
|
|
|
|
|
dev_info(&device->device, "Send section size: %d, Section count:%d\n",
|
|
|
|
net_device->send_section_size, net_device->send_section_cnt);
|
|
|
|
|
|
|
|
/* Setup state for managing the send buffer. */
|
|
|
|
net_device->map_words = DIV_ROUND_UP(net_device->send_section_cnt,
|
|
|
|
BITS_PER_LONG);
|
|
|
|
|
|
|
|
net_device->send_section_map =
|
|
|
|
kzalloc(net_device->map_words * sizeof(ulong), GFP_KERNEL);
|
2014-07-23 09:00:35 +08:00
|
|
|
if (net_device->send_section_map == NULL) {
|
|
|
|
ret = -ENOMEM;
|
2014-05-01 01:14:31 +08:00
|
|
|
goto cleanup;
|
2014-07-23 09:00:35 +08:00
|
|
|
}
|
2014-05-01 01:14:31 +08:00
|
|
|
|
2011-02-12 01:59:43 +08:00
|
|
|
goto exit;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2011-02-12 01:59:43 +08:00
|
|
|
cleanup:
|
2016-05-13 19:55:22 +08:00
|
|
|
netvsc_destroy_buf(device);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2011-02-12 01:59:43 +08:00
|
|
|
exit:
|
2009-07-14 06:34:54 +08:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2011-12-16 05:45:16 +08:00
|
|
|
/* Negotiate NVSP protocol version */
|
|
|
|
static int negotiate_nvsp_ver(struct hv_device *device,
|
|
|
|
struct netvsc_device *net_device,
|
|
|
|
struct nvsp_message *init_packet,
|
|
|
|
u32 nvsp_ver)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2016-05-13 19:55:23 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
2015-01-25 22:46:31 +08:00
|
|
|
int ret;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
memset(init_packet, 0, sizeof(struct nvsp_message));
|
2010-12-11 04:03:59 +08:00
|
|
|
init_packet->hdr.msg_type = NVSP_MSG_TYPE_INIT;
|
2011-12-16 05:45:16 +08:00
|
|
|
init_packet->msg.init_msg.init.min_protocol_ver = nvsp_ver;
|
|
|
|
init_packet->msg.init_msg.init.max_protocol_ver = nvsp_ver;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Send the init request */
|
2010-12-11 04:03:54 +08:00
|
|
|
ret = vmbus_sendpacket(device->channel, init_packet,
|
2010-10-22 00:43:24 +08:00
|
|
|
sizeof(struct nvsp_message),
|
2010-12-11 04:03:54 +08:00
|
|
|
(unsigned long)init_packet,
|
2011-01-27 04:12:13 +08:00
|
|
|
VM_PKT_DATA_INBAND,
|
2010-10-22 00:43:24 +08:00
|
|
|
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED);
|
2009-09-03 01:33:05 +08:00
|
|
|
|
2011-03-30 04:58:45 +08:00
|
|
|
if (ret != 0)
|
2011-12-16 05:45:16 +08:00
|
|
|
return ret;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2016-06-09 18:44:03 +08:00
|
|
|
wait_for_completion(&net_device->channel_init_wait);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:59 +08:00
|
|
|
if (init_packet->msg.init_msg.init_complete.status !=
|
2011-12-16 05:45:16 +08:00
|
|
|
NVSP_STAT_SUCCESS)
|
|
|
|
return -EINVAL;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2014-02-20 07:49:45 +08:00
|
|
|
if (nvsp_ver == NVSP_PROTOCOL_VERSION_1)
|
2011-12-16 05:45:16 +08:00
|
|
|
return 0;
|
|
|
|
|
2015-07-25 01:08:40 +08:00
|
|
|
/* NVSPv2 or later: Send NDIS config */
|
2011-12-16 05:45:16 +08:00
|
|
|
memset(init_packet, 0, sizeof(struct nvsp_message));
|
|
|
|
init_packet->hdr.msg_type = NVSP_MSG2_TYPE_SEND_NDIS_CONFIG;
|
2016-05-13 19:55:23 +08:00
|
|
|
init_packet->msg.v2_msg.send_ndis_config.mtu = ndev->mtu + ETH_HLEN;
|
2012-03-12 18:20:50 +08:00
|
|
|
init_packet->msg.v2_msg.send_ndis_config.capability.ieee8021q = 1;
|
2011-12-16 05:45:16 +08:00
|
|
|
|
2016-08-05 01:42:15 +08:00
|
|
|
if (nvsp_ver >= NVSP_PROTOCOL_VERSION_5) {
|
2015-07-25 01:08:40 +08:00
|
|
|
init_packet->msg.v2_msg.send_ndis_config.capability.sriov = 1;
|
|
|
|
|
2016-08-05 01:42:15 +08:00
|
|
|
/* Teaming bit is needed to receive link speed updates */
|
|
|
|
init_packet->msg.v2_msg.send_ndis_config.capability.teaming = 1;
|
|
|
|
}
|
|
|
|
|
2011-12-16 05:45:16 +08:00
|
|
|
ret = vmbus_sendpacket(device->channel, init_packet,
|
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
(unsigned long)init_packet,
|
|
|
|
VM_PKT_DATA_INBAND, 0);
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
|
|
|
static int netvsc_connect_vsp(struct hv_device *device)
|
|
|
|
{
|
|
|
|
int ret;
|
|
|
|
struct netvsc_device *net_device;
|
|
|
|
struct nvsp_message *init_packet;
|
|
|
|
int ndis_version;
|
2014-02-20 07:49:45 +08:00
|
|
|
u32 ver_list[] = { NVSP_PROTOCOL_VERSION_1, NVSP_PROTOCOL_VERSION_2,
|
|
|
|
NVSP_PROTOCOL_VERSION_4, NVSP_PROTOCOL_VERSION_5 };
|
|
|
|
int i, num_ver = 4; /* number of different NVSP versions */
|
2011-12-16 05:45:16 +08:00
|
|
|
|
|
|
|
net_device = get_outbound_net_device(device);
|
|
|
|
if (!net_device)
|
|
|
|
return -ENODEV;
|
|
|
|
|
|
|
|
init_packet = &net_device->channel_init_pkt;
|
|
|
|
|
|
|
|
/* Negotiate the latest NVSP protocol supported */
|
2014-02-20 07:49:45 +08:00
|
|
|
for (i = num_ver - 1; i >= 0; i--)
|
|
|
|
if (negotiate_nvsp_ver(device, net_device, init_packet,
|
|
|
|
ver_list[i]) == 0) {
|
|
|
|
net_device->nvsp_version = ver_list[i];
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (i < 0) {
|
2011-08-26 00:49:14 +08:00
|
|
|
ret = -EPROTO;
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
2011-12-16 05:45:16 +08:00
|
|
|
|
|
|
|
pr_debug("Negotiated NVSP version:%x\n", net_device->nvsp_version);
|
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Send the ndis version */
|
2010-12-11 04:03:54 +08:00
|
|
|
memset(init_packet, 0, sizeof(struct nvsp_message));
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2014-02-20 07:49:45 +08:00
|
|
|
if (net_device->nvsp_version <= NVSP_PROTOCOL_VERSION_4)
|
2014-04-10 06:00:46 +08:00
|
|
|
ndis_version = 0x00060001;
|
2014-02-20 07:49:45 +08:00
|
|
|
else
|
|
|
|
ndis_version = 0x0006001e;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:59 +08:00
|
|
|
init_packet->hdr.msg_type = NVSP_MSG1_TYPE_SEND_NDIS_VER;
|
|
|
|
init_packet->msg.v1_msg.
|
|
|
|
send_ndis_ver.ndis_major_ver =
|
2010-12-11 04:03:54 +08:00
|
|
|
(ndis_version & 0xFFFF0000) >> 16;
|
2010-12-11 04:03:59 +08:00
|
|
|
init_packet->msg.v1_msg.
|
|
|
|
send_ndis_ver.ndis_minor_ver =
|
2010-12-11 04:03:54 +08:00
|
|
|
ndis_version & 0xFFFF;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Send the init request */
|
2010-12-11 04:03:54 +08:00
|
|
|
ret = vmbus_sendpacket(device->channel, init_packet,
|
2011-02-12 01:59:43 +08:00
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
(unsigned long)init_packet,
|
|
|
|
VM_PKT_DATA_INBAND, 0);
|
2011-08-26 00:49:14 +08:00
|
|
|
if (ret != 0)
|
2011-02-12 01:59:43 +08:00
|
|
|
goto cleanup;
|
2009-07-28 04:47:24 +08:00
|
|
|
|
|
|
|
/* Post the big receive buffer to NetVSP */
|
2014-03-10 07:10:59 +08:00
|
|
|
if (net_device->nvsp_version <= NVSP_PROTOCOL_VERSION_2)
|
|
|
|
net_device->recv_buf_size = NETVSC_RECEIVE_BUFFER_SIZE_LEGACY;
|
|
|
|
else
|
|
|
|
net_device->recv_buf_size = NETVSC_RECEIVE_BUFFER_SIZE;
|
2014-05-01 01:14:31 +08:00
|
|
|
net_device->send_buf_size = NETVSC_SEND_BUFFER_SIZE;
|
2014-03-10 07:10:59 +08:00
|
|
|
|
2014-05-01 01:14:31 +08:00
|
|
|
ret = netvsc_init_buf(device);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2011-02-12 01:59:43 +08:00
|
|
|
cleanup:
|
2009-07-14 06:34:54 +08:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2016-05-13 19:55:22 +08:00
|
|
|
static void netvsc_disconnect_vsp(struct hv_device *device)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2016-05-13 19:55:22 +08:00
|
|
|
netvsc_destroy_buf(device);
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2010-03-05 06:11:00 +08:00
|
|
|
/*
|
2010-12-11 04:03:55 +08:00
|
|
|
* netvsc_device_remove - Callback when the root bus device is removed
|
2009-09-03 01:33:05 +08:00
|
|
|
*/
|
2011-05-10 22:54:54 +08:00
|
|
|
int netvsc_device_remove(struct hv_device *device)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2016-05-13 19:55:22 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
|
|
|
struct net_device_context *net_device_ctx = netdev_priv(ndev);
|
|
|
|
struct netvsc_device *net_device = net_device_ctx->nvdev;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2016-05-13 19:55:22 +08:00
|
|
|
netvsc_disconnect_vsp(device);
|
2011-08-28 02:31:16 +08:00
|
|
|
|
2016-05-13 19:55:22 +08:00
|
|
|
net_device_ctx->nvdev = NULL;
|
2011-08-28 02:31:14 +08:00
|
|
|
|
2011-09-14 01:59:54 +08:00
|
|
|
/*
|
|
|
|
* At this point, no one should be accessing net_device
|
|
|
|
* except in here
|
|
|
|
*/
|
2011-09-02 03:19:40 +08:00
|
|
|
dev_notice(&device->device, "net device safe to remove\n");
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Now, we can close the channel safely */
|
2010-12-11 04:03:54 +08:00
|
|
|
vmbus_close(device->channel);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Release all resources */
|
2014-11-26 05:33:45 +08:00
|
|
|
vfree(net_device->sub_cb_buf);
|
2014-08-16 03:18:19 +08:00
|
|
|
free_netvsc_device(net_device);
|
2009-09-03 01:33:05 +08:00
|
|
|
return 0;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2012-03-27 21:20:45 +08:00
|
|
|
|
|
|
|
#define RING_AVAIL_PERCENT_HIWATER 20
|
|
|
|
#define RING_AVAIL_PERCENT_LOWATER 10
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Get the percentage of available bytes to write in the ring.
|
|
|
|
* The return value is in range from 0 to 100.
|
|
|
|
*/
|
|
|
|
static inline u32 hv_ringbuf_avail_percent(
|
|
|
|
struct hv_ring_buffer_info *ring_info)
|
|
|
|
{
|
|
|
|
u32 avail_read, avail_write;
|
|
|
|
|
|
|
|
hv_get_ringbuffer_availbytes(ring_info, &avail_read, &avail_write);
|
|
|
|
|
|
|
|
return avail_write * 100 / ring_info->ring_datasize;
|
|
|
|
}
|
|
|
|
|
2014-05-01 01:14:31 +08:00
|
|
|
static inline void netvsc_free_send_slot(struct netvsc_device *net_device,
|
|
|
|
u32 index)
|
|
|
|
{
|
|
|
|
sync_change_bit(index, net_device->send_section_map);
|
|
|
|
}
|
|
|
|
|
2014-02-17 08:38:44 +08:00
|
|
|
static void netvsc_send_completion(struct netvsc_device *net_device,
|
2015-12-02 08:43:05 +08:00
|
|
|
struct vmbus_channel *incoming_channel,
|
2014-02-17 08:38:44 +08:00
|
|
|
struct hv_device *device,
|
2010-12-11 04:03:54 +08:00
|
|
|
struct vmpacket_descriptor *packet)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2010-12-11 04:03:54 +08:00
|
|
|
struct nvsp_message *nvsp_packet;
|
|
|
|
struct hv_netvsc_packet *nvsc_packet;
|
2016-05-13 19:55:22 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
|
|
|
struct net_device_context *net_device_ctx = netdev_priv(ndev);
|
2014-05-01 01:14:31 +08:00
|
|
|
u32 send_index;
|
2015-12-02 08:43:14 +08:00
|
|
|
struct sk_buff *skb;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
nvsp_packet = (struct nvsp_message *)((unsigned long)packet +
|
2011-01-27 04:12:13 +08:00
|
|
|
(packet->offset8 << 3));
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2010-12-11 04:03:59 +08:00
|
|
|
if ((nvsp_packet->hdr.msg_type == NVSP_MSG_TYPE_INIT_COMPLETE) ||
|
|
|
|
(nvsp_packet->hdr.msg_type ==
|
|
|
|
NVSP_MSG1_TYPE_SEND_RECV_BUF_COMPLETE) ||
|
|
|
|
(nvsp_packet->hdr.msg_type ==
|
2014-04-22 01:20:28 +08:00
|
|
|
NVSP_MSG1_TYPE_SEND_SEND_BUF_COMPLETE) ||
|
|
|
|
(nvsp_packet->hdr.msg_type ==
|
|
|
|
NVSP_MSG5_TYPE_SUBCHANNEL)) {
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Copy the response back */
|
2010-12-11 04:03:59 +08:00
|
|
|
memcpy(&net_device->channel_init_pkt, nvsp_packet,
|
2009-09-03 01:33:05 +08:00
|
|
|
sizeof(struct nvsp_message));
|
2011-05-10 22:55:41 +08:00
|
|
|
complete(&net_device->channel_init_wait);
|
2010-12-11 04:03:59 +08:00
|
|
|
} else if (nvsp_packet->hdr.msg_type ==
|
|
|
|
NVSP_MSG1_TYPE_SEND_RNDIS_PKT_COMPLETE) {
|
2012-03-27 21:20:45 +08:00
|
|
|
int num_outstanding_sends;
|
2014-04-22 01:20:28 +08:00
|
|
|
u16 q_idx = 0;
|
|
|
|
struct vmbus_channel *channel = device->channel;
|
|
|
|
int queue_sends;
|
2012-03-27 21:20:45 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Get the send context */
|
2015-12-02 08:43:14 +08:00
|
|
|
skb = (struct sk_buff *)(unsigned long)packet->trans_id;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Notify the layer above us */
|
2015-12-02 08:43:14 +08:00
|
|
|
if (skb) {
|
|
|
|
nvsc_packet = (struct hv_netvsc_packet *) skb->cb;
|
2014-05-01 01:14:31 +08:00
|
|
|
send_index = nvsc_packet->send_buf_index;
|
|
|
|
if (send_index != NETVSC_INVALID_INDEX)
|
|
|
|
netvsc_free_send_slot(net_device, send_index);
|
2014-04-22 01:20:28 +08:00
|
|
|
q_idx = nvsc_packet->q_idx;
|
2015-12-02 08:43:05 +08:00
|
|
|
channel = incoming_channel;
|
2015-12-02 08:43:14 +08:00
|
|
|
dev_kfree_skb_any(skb);
|
2014-04-22 01:20:28 +08:00
|
|
|
}
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2012-03-27 21:20:45 +08:00
|
|
|
num_outstanding_sends =
|
|
|
|
atomic_dec_return(&net_device->num_outstanding_sends);
|
2014-04-22 01:20:28 +08:00
|
|
|
queue_sends = atomic_dec_return(&net_device->
|
|
|
|
queue_sends[q_idx]);
|
2011-12-03 03:56:25 +08:00
|
|
|
|
2012-06-04 14:42:38 +08:00
|
|
|
if (net_device->destroy && num_outstanding_sends == 0)
|
|
|
|
wake_up(&net_device->wait_drain);
|
|
|
|
|
2014-04-22 01:20:28 +08:00
|
|
|
if (netif_tx_queue_stopped(netdev_get_tx_queue(ndev, q_idx)) &&
|
2016-05-13 19:55:22 +08:00
|
|
|
!net_device_ctx->start_remove &&
|
2014-04-22 01:20:28 +08:00
|
|
|
(hv_ringbuf_avail_percent(&channel->outbound) >
|
|
|
|
RING_AVAIL_PERCENT_HIWATER || queue_sends < 1))
|
|
|
|
netif_tx_wake_queue(netdev_get_tx_queue(
|
|
|
|
ndev, q_idx));
|
2009-09-03 01:33:05 +08:00
|
|
|
} else {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "Unknown send completion packet type- "
|
2011-09-02 03:19:40 +08:00
|
|
|
"%d received!!\n", nvsp_packet->hdr.msg_type);
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
|
|
|
}
|
|
|
|
|
2014-05-01 01:14:31 +08:00
|
|
|
static u32 netvsc_get_next_send_section(struct netvsc_device *net_device)
|
|
|
|
{
|
|
|
|
unsigned long index;
|
|
|
|
u32 max_words = net_device->map_words;
|
|
|
|
unsigned long *map_addr = (unsigned long *)net_device->send_section_map;
|
|
|
|
u32 section_cnt = net_device->send_section_cnt;
|
|
|
|
int ret_val = NETVSC_INVALID_INDEX;
|
|
|
|
int i;
|
|
|
|
int prev_val;
|
|
|
|
|
|
|
|
for (i = 0; i < max_words; i++) {
|
|
|
|
if (!~(map_addr[i]))
|
|
|
|
continue;
|
|
|
|
index = ffz(map_addr[i]);
|
|
|
|
prev_val = sync_test_and_set_bit(index, &map_addr[i]);
|
|
|
|
if (prev_val)
|
|
|
|
continue;
|
|
|
|
if ((index + (i * BITS_PER_LONG)) >= section_cnt)
|
|
|
|
break;
|
|
|
|
ret_val = (index + (i * BITS_PER_LONG));
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
return ret_val;
|
|
|
|
}
|
|
|
|
|
2015-02-05 23:06:33 +08:00
|
|
|
static u32 netvsc_copy_to_send_buf(struct netvsc_device *net_device,
|
|
|
|
unsigned int section_index,
|
2015-03-27 00:03:37 +08:00
|
|
|
u32 pend_size,
|
2015-12-02 08:43:06 +08:00
|
|
|
struct hv_netvsc_packet *packet,
|
2015-12-02 08:43:13 +08:00
|
|
|
struct rndis_message *rndis_msg,
|
2015-12-02 08:43:15 +08:00
|
|
|
struct hv_page_buffer **pb,
|
|
|
|
struct sk_buff *skb)
|
2014-05-01 01:14:31 +08:00
|
|
|
{
|
|
|
|
char *start = net_device->send_buf;
|
2015-03-27 00:03:37 +08:00
|
|
|
char *dest = start + (section_index * net_device->send_section_size)
|
|
|
|
+ pend_size;
|
2014-05-01 01:14:31 +08:00
|
|
|
int i;
|
2015-12-02 08:43:15 +08:00
|
|
|
bool is_data_pkt = (skb != NULL) ? true : false;
|
2015-12-02 08:43:17 +08:00
|
|
|
bool xmit_more = (skb != NULL) ? skb->xmit_more : false;
|
2014-05-01 01:14:31 +08:00
|
|
|
u32 msg_size = 0;
|
2015-03-27 00:03:37 +08:00
|
|
|
u32 padding = 0;
|
|
|
|
u32 remain = packet->total_data_buflen % net_device->pkt_align;
|
2015-04-14 07:34:35 +08:00
|
|
|
u32 page_count = packet->cp_partial ? packet->rmsg_pgcnt :
|
|
|
|
packet->page_buf_cnt;
|
2015-03-27 00:03:37 +08:00
|
|
|
|
|
|
|
/* Add padding */
|
2015-12-02 08:43:17 +08:00
|
|
|
if (is_data_pkt && xmit_more && remain &&
|
2015-04-14 07:34:35 +08:00
|
|
|
!packet->cp_partial) {
|
2015-03-27 00:03:37 +08:00
|
|
|
padding = net_device->pkt_align - remain;
|
2015-12-02 08:43:06 +08:00
|
|
|
rndis_msg->msg_len += padding;
|
2015-03-27 00:03:37 +08:00
|
|
|
packet->total_data_buflen += padding;
|
|
|
|
}
|
2014-05-01 01:14:31 +08:00
|
|
|
|
2015-04-14 07:34:35 +08:00
|
|
|
for (i = 0; i < page_count; i++) {
|
2015-12-02 08:43:13 +08:00
|
|
|
char *src = phys_to_virt((*pb)[i].pfn << PAGE_SHIFT);
|
|
|
|
u32 offset = (*pb)[i].offset;
|
|
|
|
u32 len = (*pb)[i].len;
|
2014-05-01 01:14:31 +08:00
|
|
|
|
|
|
|
memcpy(dest, (src + offset), len);
|
|
|
|
msg_size += len;
|
|
|
|
dest += len;
|
|
|
|
}
|
2015-03-27 00:03:37 +08:00
|
|
|
|
|
|
|
if (padding) {
|
|
|
|
memset(dest, 0, padding);
|
|
|
|
msg_size += padding;
|
|
|
|
}
|
|
|
|
|
2014-05-01 01:14:31 +08:00
|
|
|
return msg_size;
|
|
|
|
}
|
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
static inline int netvsc_send_pkt(
|
2016-05-13 19:55:23 +08:00
|
|
|
struct hv_device *device,
|
2015-03-27 00:03:37 +08:00
|
|
|
struct hv_netvsc_packet *packet,
|
2015-12-02 08:43:13 +08:00
|
|
|
struct netvsc_device *net_device,
|
2015-12-02 08:43:14 +08:00
|
|
|
struct hv_page_buffer **pb,
|
|
|
|
struct sk_buff *skb)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2015-03-27 00:03:37 +08:00
|
|
|
struct nvsp_message nvmsg;
|
2014-10-06 01:42:51 +08:00
|
|
|
u16 q_idx = packet->q_idx;
|
2015-12-02 08:43:11 +08:00
|
|
|
struct vmbus_channel *out_channel = net_device->chn_table[q_idx];
|
2016-05-13 19:55:23 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
2015-03-27 00:03:37 +08:00
|
|
|
u64 req_id;
|
|
|
|
int ret;
|
2015-04-14 07:34:35 +08:00
|
|
|
struct hv_page_buffer *pgbuf;
|
2015-05-12 06:39:46 +08:00
|
|
|
u32 ring_avail = hv_ringbuf_avail_percent(&out_channel->outbound);
|
2015-12-02 08:43:17 +08:00
|
|
|
bool xmit_more = (skb != NULL) ? skb->xmit_more : false;
|
2014-05-01 01:14:31 +08:00
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
nvmsg.hdr.msg_type = NVSP_MSG1_TYPE_SEND_RNDIS_PKT;
|
2015-12-02 08:43:15 +08:00
|
|
|
if (skb != NULL) {
|
2009-09-03 01:33:05 +08:00
|
|
|
/* 0 is RMC_DATA; */
|
2015-03-27 00:03:37 +08:00
|
|
|
nvmsg.msg.v1_msg.send_rndis_pkt.channel_type = 0;
|
2009-09-03 01:33:05 +08:00
|
|
|
} else {
|
|
|
|
/* 1 is RMC_CONTROL; */
|
2015-03-27 00:03:37 +08:00
|
|
|
nvmsg.msg.v1_msg.send_rndis_pkt.channel_type = 1;
|
2009-09-03 01:33:05 +08:00
|
|
|
}
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
nvmsg.msg.v1_msg.send_rndis_pkt.send_buf_section_index =
|
|
|
|
packet->send_buf_index;
|
|
|
|
if (packet->send_buf_index == NETVSC_INVALID_INDEX)
|
|
|
|
nvmsg.msg.v1_msg.send_rndis_pkt.send_buf_section_size = 0;
|
|
|
|
else
|
|
|
|
nvmsg.msg.v1_msg.send_rndis_pkt.send_buf_section_size =
|
|
|
|
packet->total_data_buflen;
|
2009-09-03 01:33:05 +08:00
|
|
|
|
2015-12-02 08:43:14 +08:00
|
|
|
req_id = (ulong)skb;
|
2013-04-05 19:44:40 +08:00
|
|
|
|
2014-12-02 05:28:39 +08:00
|
|
|
if (out_channel->rescind)
|
|
|
|
return -ENODEV;
|
|
|
|
|
2015-05-12 06:39:46 +08:00
|
|
|
/*
|
|
|
|
* It is possible that once we successfully place this packet
|
|
|
|
* on the ringbuffer, we may stop the queue. In that case, we want
|
|
|
|
* to notify the host independent of the xmit_more flag. We don't
|
|
|
|
* need to be precise here; in the worst case we may signal the host
|
|
|
|
* unnecessarily.
|
|
|
|
*/
|
|
|
|
if (ring_avail < (RING_AVAIL_PERCENT_LOWATER + 1))
|
2015-12-02 08:43:17 +08:00
|
|
|
xmit_more = false;
|
2015-05-12 06:39:46 +08:00
|
|
|
|
2010-12-11 04:03:58 +08:00
|
|
|
if (packet->page_buf_cnt) {
|
2015-12-02 08:43:13 +08:00
|
|
|
pgbuf = packet->cp_partial ? (*pb) +
|
|
|
|
packet->rmsg_pgcnt : (*pb);
|
2015-05-12 06:39:46 +08:00
|
|
|
ret = vmbus_sendpacket_pagebuffer_ctl(out_channel,
|
|
|
|
pgbuf,
|
|
|
|
packet->page_buf_cnt,
|
|
|
|
&nvmsg,
|
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
req_id,
|
|
|
|
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED,
|
2015-12-02 08:43:17 +08:00
|
|
|
!xmit_more);
|
2009-09-03 01:33:05 +08:00
|
|
|
} else {
|
2015-05-12 06:39:46 +08:00
|
|
|
ret = vmbus_sendpacket_ctl(out_channel, &nvmsg,
|
|
|
|
sizeof(struct nvsp_message),
|
|
|
|
req_id,
|
|
|
|
VM_PKT_DATA_INBAND,
|
|
|
|
VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED,
|
2015-12-02 08:43:17 +08:00
|
|
|
!xmit_more);
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2011-12-03 03:56:25 +08:00
|
|
|
if (ret == 0) {
|
|
|
|
atomic_inc(&net_device->num_outstanding_sends);
|
2014-10-06 01:42:51 +08:00
|
|
|
atomic_inc(&net_device->queue_sends[q_idx]);
|
2014-04-22 01:20:28 +08:00
|
|
|
|
2015-05-12 06:39:46 +08:00
|
|
|
if (ring_avail < RING_AVAIL_PERCENT_LOWATER) {
|
|
|
|
netif_tx_stop_queue(netdev_get_tx_queue(ndev, q_idx));
|
2014-04-22 01:20:28 +08:00
|
|
|
|
2012-03-27 21:20:45 +08:00
|
|
|
if (atomic_read(&net_device->
|
2014-10-06 01:42:51 +08:00
|
|
|
queue_sends[q_idx]) < 1)
|
2014-04-22 01:20:28 +08:00
|
|
|
netif_tx_wake_queue(netdev_get_tx_queue(
|
2014-10-06 01:42:51 +08:00
|
|
|
ndev, q_idx));
|
2012-03-27 21:20:45 +08:00
|
|
|
}
|
2011-12-03 03:56:25 +08:00
|
|
|
} else if (ret == -EAGAIN) {
|
2014-04-22 01:20:28 +08:00
|
|
|
netif_tx_stop_queue(netdev_get_tx_queue(
|
2014-10-06 01:42:51 +08:00
|
|
|
ndev, q_idx));
|
|
|
|
if (atomic_read(&net_device->queue_sends[q_idx]) < 1) {
|
2014-04-22 01:20:28 +08:00
|
|
|
netif_tx_wake_queue(netdev_get_tx_queue(
|
2014-10-06 01:42:51 +08:00
|
|
|
ndev, q_idx));
|
2012-03-27 21:20:45 +08:00
|
|
|
ret = -ENOSPC;
|
|
|
|
}
|
2011-12-03 03:56:25 +08:00
|
|
|
} else {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "Unable to send packet %p ret %d\n",
|
2010-12-11 04:03:54 +08:00
|
|
|
packet, ret);
|
2011-12-03 03:56:25 +08:00
|
|
|
}
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2016-01-26 01:49:31 +08:00
|
|
|
/* Move packet out of multi send data (msd), and clear msd */
|
|
|
|
static inline void move_pkt_msd(struct hv_netvsc_packet **msd_send,
|
|
|
|
struct sk_buff **msd_skb,
|
|
|
|
struct multi_send_data *msdp)
|
|
|
|
{
|
|
|
|
*msd_skb = msdp->skb;
|
|
|
|
*msd_send = msdp->pkt;
|
|
|
|
msdp->skb = NULL;
|
|
|
|
msdp->pkt = NULL;
|
|
|
|
msdp->count = 0;
|
|
|
|
}
|
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
int netvsc_send(struct hv_device *device,
|
2015-12-02 08:43:06 +08:00
|
|
|
struct hv_netvsc_packet *packet,
|
2015-12-02 08:43:13 +08:00
|
|
|
struct rndis_message *rndis_msg,
|
2015-12-02 08:43:14 +08:00
|
|
|
struct hv_page_buffer **pb,
|
|
|
|
struct sk_buff *skb)
|
2015-03-27 00:03:37 +08:00
|
|
|
{
|
|
|
|
struct netvsc_device *net_device;
|
|
|
|
int ret = 0, m_ret = 0;
|
|
|
|
struct vmbus_channel *out_channel;
|
|
|
|
u16 q_idx = packet->q_idx;
|
|
|
|
u32 pktlen = packet->total_data_buflen, msd_len = 0;
|
|
|
|
unsigned int section_index = NETVSC_INVALID_INDEX;
|
|
|
|
struct multi_send_data *msdp;
|
|
|
|
struct hv_netvsc_packet *msd_send = NULL, *cur_send = NULL;
|
2016-01-26 01:49:31 +08:00
|
|
|
struct sk_buff *msd_skb = NULL;
|
2015-04-14 07:34:35 +08:00
|
|
|
bool try_batch;
|
2015-12-02 08:43:17 +08:00
|
|
|
bool xmit_more = (skb != NULL) ? skb->xmit_more : false;
|
2015-03-27 00:03:37 +08:00
|
|
|
|
|
|
|
net_device = get_outbound_net_device(device);
|
|
|
|
if (!net_device)
|
|
|
|
return -ENODEV;
|
|
|
|
|
2015-12-02 08:43:11 +08:00
|
|
|
out_channel = net_device->chn_table[q_idx];
|
2015-12-02 08:43:05 +08:00
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
packet->send_buf_index = NETVSC_INVALID_INDEX;
|
2015-04-14 07:34:35 +08:00
|
|
|
packet->cp_partial = false;
|
2015-03-27 00:03:37 +08:00
|
|
|
|
2015-12-11 04:19:35 +08:00
|
|
|
/* Send control message directly without accessing msd (Multi-Send
|
|
|
|
* Data) field which may be changed during data packet processing.
|
|
|
|
*/
|
|
|
|
if (!skb) {
|
|
|
|
cur_send = packet;
|
|
|
|
goto send_now;
|
|
|
|
}
|
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
msdp = &net_device->msd[q_idx];
|
|
|
|
|
|
|
|
/* batch packets in send buffer if possible */
|
|
|
|
if (msdp->pkt)
|
|
|
|
msd_len = msdp->pkt->total_data_buflen;
|
|
|
|
|
2015-12-02 08:43:15 +08:00
|
|
|
try_batch = (skb != NULL) && msd_len > 0 && msdp->count <
|
2015-04-14 07:34:35 +08:00
|
|
|
net_device->max_pkt;
|
|
|
|
|
|
|
|
if (try_batch && msd_len + pktlen + net_device->pkt_align <
|
2015-03-27 00:03:37 +08:00
|
|
|
net_device->send_section_size) {
|
|
|
|
section_index = msdp->pkt->send_buf_index;
|
|
|
|
|
2015-04-14 07:34:35 +08:00
|
|
|
} else if (try_batch && msd_len + packet->rmsg_size <
|
|
|
|
net_device->send_section_size) {
|
|
|
|
section_index = msdp->pkt->send_buf_index;
|
|
|
|
packet->cp_partial = true;
|
|
|
|
|
2015-12-02 08:43:15 +08:00
|
|
|
} else if ((skb != NULL) && pktlen + net_device->pkt_align <
|
2015-03-27 00:03:37 +08:00
|
|
|
net_device->send_section_size) {
|
|
|
|
section_index = netvsc_get_next_send_section(net_device);
|
|
|
|
if (section_index != NETVSC_INVALID_INDEX) {
|
2016-01-26 01:49:31 +08:00
|
|
|
move_pkt_msd(&msd_send, &msd_skb, msdp);
|
|
|
|
msd_len = 0;
|
2015-03-27 00:03:37 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if (section_index != NETVSC_INVALID_INDEX) {
|
|
|
|
netvsc_copy_to_send_buf(net_device,
|
|
|
|
section_index, msd_len,
|
2015-12-02 08:43:15 +08:00
|
|
|
packet, rndis_msg, pb, skb);
|
2015-03-30 12:08:42 +08:00
|
|
|
|
2015-03-27 00:03:37 +08:00
|
|
|
packet->send_buf_index = section_index;
|
2015-04-14 07:34:35 +08:00
|
|
|
|
|
|
|
if (packet->cp_partial) {
|
|
|
|
packet->page_buf_cnt -= packet->rmsg_pgcnt;
|
|
|
|
packet->total_data_buflen = msd_len + packet->rmsg_size;
|
|
|
|
} else {
|
|
|
|
packet->page_buf_cnt = 0;
|
|
|
|
packet->total_data_buflen += msd_len;
|
|
|
|
}
|
2015-03-27 00:03:37 +08:00
|
|
|
|
2016-01-26 01:49:31 +08:00
|
|
|
if (msdp->skb)
|
|
|
|
dev_kfree_skb_any(msdp->skb);
|
2015-04-07 06:22:54 +08:00
|
|
|
|
2015-12-02 08:43:17 +08:00
|
|
|
if (xmit_more && !packet->cp_partial) {
|
2016-01-26 01:49:31 +08:00
|
|
|
msdp->skb = skb;
|
2015-03-27 00:03:37 +08:00
|
|
|
msdp->pkt = packet;
|
|
|
|
msdp->count++;
|
|
|
|
} else {
|
|
|
|
cur_send = packet;
|
2016-01-26 01:49:31 +08:00
|
|
|
msdp->skb = NULL;
|
2015-03-27 00:03:37 +08:00
|
|
|
msdp->pkt = NULL;
|
|
|
|
msdp->count = 0;
|
|
|
|
}
|
|
|
|
} else {
|
2016-01-26 01:49:31 +08:00
|
|
|
move_pkt_msd(&msd_send, &msd_skb, msdp);
|
2015-03-27 00:03:37 +08:00
|
|
|
cur_send = packet;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (msd_send) {
|
2016-05-13 19:55:23 +08:00
|
|
|
m_ret = netvsc_send_pkt(device, msd_send, net_device,
|
|
|
|
NULL, msd_skb);
|
2015-03-27 00:03:37 +08:00
|
|
|
|
|
|
|
if (m_ret != 0) {
|
|
|
|
netvsc_free_send_slot(net_device,
|
|
|
|
msd_send->send_buf_index);
|
2016-01-26 01:49:31 +08:00
|
|
|
dev_kfree_skb_any(msd_skb);
|
2015-03-27 00:03:37 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2015-12-11 04:19:35 +08:00
|
|
|
send_now:
|
2015-03-27 00:03:37 +08:00
|
|
|
if (cur_send)
|
2016-05-13 19:55:23 +08:00
|
|
|
ret = netvsc_send_pkt(device, cur_send, net_device, pb, skb);
|
2015-03-27 00:03:37 +08:00
|
|
|
|
2015-05-05 01:57:16 +08:00
|
|
|
if (ret != 0 && section_index != NETVSC_INVALID_INDEX)
|
|
|
|
netvsc_free_send_slot(net_device, section_index);
|
2015-01-30 04:34:49 +08:00
|
|
|
|
2009-07-14 06:34:54 +08:00
|
|
|
return ret;
|
|
|
|
}
|
|
|
|
|
2011-04-22 03:30:42 +08:00
|
|
|
static void netvsc_send_recv_completion(struct hv_device *device,
|
2014-04-22 01:20:28 +08:00
|
|
|
struct vmbus_channel *channel,
|
2014-02-17 08:38:44 +08:00
|
|
|
struct netvsc_device *net_device,
|
2012-10-02 13:30:23 +08:00
|
|
|
u64 transaction_id, u32 status)
|
2011-04-22 03:30:42 +08:00
|
|
|
{
|
|
|
|
struct nvsp_message recvcompMessage;
|
|
|
|
int retries = 0;
|
|
|
|
int ret;
|
2016-05-13 19:55:23 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
2011-04-22 03:30:42 +08:00
|
|
|
|
|
|
|
recvcompMessage.hdr.msg_type =
|
|
|
|
NVSP_MSG1_TYPE_SEND_RNDIS_PKT_COMPLETE;
|
|
|
|
|
2012-10-02 13:30:23 +08:00
|
|
|
recvcompMessage.msg.v1_msg.send_rndis_pkt_complete.status = status;
|
2011-04-22 03:30:42 +08:00
|
|
|
|
|
|
|
retry_send_cmplt:
|
|
|
|
/* Send the completion */
|
2014-04-22 01:20:28 +08:00
|
|
|
ret = vmbus_sendpacket(channel, &recvcompMessage,
|
2011-04-22 03:30:42 +08:00
|
|
|
sizeof(struct nvsp_message), transaction_id,
|
|
|
|
VM_PKT_COMP, 0);
|
|
|
|
if (ret == 0) {
|
|
|
|
/* success */
|
|
|
|
/* no-op */
|
2011-08-26 00:48:58 +08:00
|
|
|
} else if (ret == -EAGAIN) {
|
2011-04-22 03:30:42 +08:00
|
|
|
/* no more room...wait a bit and attempt to retry 3 times */
|
|
|
|
retries++;
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "unable to send receive completion pkt"
|
2011-09-02 03:19:40 +08:00
|
|
|
" (tid %llx)...retrying %d\n", transaction_id, retries);
|
2011-04-22 03:30:42 +08:00
|
|
|
|
|
|
|
if (retries < 4) {
|
|
|
|
udelay(100);
|
|
|
|
goto retry_send_cmplt;
|
|
|
|
} else {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "unable to send receive "
|
2011-09-02 03:19:40 +08:00
|
|
|
"completion pkt (tid %llx)...give up retrying\n",
|
2011-04-22 03:30:42 +08:00
|
|
|
transaction_id);
|
|
|
|
}
|
|
|
|
} else {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "unable to send receive "
|
2011-09-02 03:19:40 +08:00
|
|
|
"completion pkt - %llx\n", transaction_id);
|
2011-04-22 03:30:42 +08:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2014-02-17 08:38:44 +08:00
|
|
|
static void netvsc_receive(struct netvsc_device *net_device,
|
2014-04-22 01:20:28 +08:00
|
|
|
struct vmbus_channel *channel,
|
2014-02-17 08:38:44 +08:00
|
|
|
struct hv_device *device,
|
|
|
|
struct vmpacket_descriptor *packet)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2010-12-11 04:03:54 +08:00
|
|
|
struct vmtransfer_page_packet_header *vmxferpage_packet;
|
|
|
|
struct nvsp_message *nvsp_packet;
|
2014-04-22 05:54:43 +08:00
|
|
|
struct hv_netvsc_packet nv_pkt;
|
|
|
|
struct hv_netvsc_packet *netvsc_packet = &nv_pkt;
|
|
|
|
u32 status = NVSP_STAT_SUCCESS;
|
2011-12-16 05:45:15 +08:00
|
|
|
int i;
|
|
|
|
int count = 0;
|
2016-05-13 19:55:23 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
2015-12-02 08:43:07 +08:00
|
|
|
void *data;
|
2011-04-27 00:20:22 +08:00
|
|
|
|
2009-09-03 01:33:05 +08:00
|
|
|
/*
|
|
|
|
* All inbound packets other than send completion should be xfer page
|
|
|
|
* packet
|
|
|
|
*/
|
2011-01-27 04:12:13 +08:00
|
|
|
if (packet->type != VM_PKT_DATA_USING_XFER_PAGES) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "Unknown packet type received - %d\n",
|
2011-01-27 04:12:13 +08:00
|
|
|
packet->type);
|
2009-07-14 06:34:54 +08:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
nvsp_packet = (struct nvsp_message *)((unsigned long)packet +
|
2011-01-27 04:12:13 +08:00
|
|
|
(packet->offset8 << 3));
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Make sure this is a valid nvsp packet */
|
2010-12-11 04:03:59 +08:00
|
|
|
if (nvsp_packet->hdr.msg_type !=
|
|
|
|
NVSP_MSG1_TYPE_SEND_RNDIS_PKT) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "Unknown nvsp packet type received-"
|
2011-09-02 03:19:40 +08:00
|
|
|
" %d\n", nvsp_packet->hdr.msg_type);
|
2009-07-14 06:34:54 +08:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
vmxferpage_packet = (struct vmtransfer_page_packet_header *)packet;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2011-01-27 04:12:13 +08:00
|
|
|
if (vmxferpage_packet->xfer_pageset_id != NETVSC_RECEIVE_BUFFER_ID) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "Invalid xfer page set id - "
|
2011-09-02 03:19:40 +08:00
|
|
|
"expecting %x got %x\n", NETVSC_RECEIVE_BUFFER_ID,
|
2011-01-27 04:12:13 +08:00
|
|
|
vmxferpage_packet->xfer_pageset_id);
|
2009-07-14 06:34:54 +08:00
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2014-04-22 05:54:43 +08:00
|
|
|
count = vmxferpage_packet->range_cnt;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Each range represents 1 RNDIS pkt that contains 1 ethernet frame */
|
2014-04-22 05:54:43 +08:00
|
|
|
for (i = 0; i < count; i++) {
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Initialize the netvsc packet */
|
2015-12-02 08:43:07 +08:00
|
|
|
data = (void *)((unsigned long)net_device->
|
2011-12-16 05:45:15 +08:00
|
|
|
recv_buf + vmxferpage_packet->ranges[i].byte_offset);
|
2010-12-11 04:03:58 +08:00
|
|
|
netvsc_packet->total_data_buflen =
|
2011-01-27 04:12:13 +08:00
|
|
|
vmxferpage_packet->ranges[i].byte_count;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Pass it to the upper layer */
|
2015-12-02 08:43:18 +08:00
|
|
|
status = rndis_filter_receive(device, netvsc_packet, &data,
|
|
|
|
channel);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
|
|
|
}
|
|
|
|
|
2014-04-22 05:54:43 +08:00
|
|
|
netvsc_send_recv_completion(device, channel, net_device,
|
|
|
|
vmxferpage_packet->d.trans_id, status);
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
|
2014-04-22 01:20:28 +08:00
|
|
|
|
|
|
|
static void netvsc_send_table(struct hv_device *hdev,
|
2015-07-25 01:08:40 +08:00
|
|
|
struct nvsp_message *nvmsg)
|
2014-04-22 01:20:28 +08:00
|
|
|
{
|
|
|
|
struct netvsc_device *nvscdev;
|
2016-05-13 19:55:23 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(hdev);
|
2014-04-22 01:20:28 +08:00
|
|
|
int i;
|
|
|
|
u32 count, *tab;
|
|
|
|
|
|
|
|
nvscdev = get_outbound_net_device(hdev);
|
|
|
|
if (!nvscdev)
|
|
|
|
return;
|
|
|
|
|
|
|
|
count = nvmsg->msg.v5_msg.send_table.count;
|
|
|
|
if (count != VRSS_SEND_TAB_SIZE) {
|
|
|
|
netdev_err(ndev, "Received wrong send-table size:%u\n", count);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
tab = (u32 *)((unsigned long)&nvmsg->msg.v5_msg.send_table +
|
|
|
|
nvmsg->msg.v5_msg.send_table.offset);
|
|
|
|
|
|
|
|
for (i = 0; i < count; i++)
|
|
|
|
nvscdev->send_table[i] = tab[i];
|
|
|
|
}
|
|
|
|
|
2015-07-25 01:08:40 +08:00
|
|
|
static void netvsc_send_vf(struct netvsc_device *nvdev,
|
|
|
|
struct nvsp_message *nvmsg)
|
|
|
|
{
|
|
|
|
nvdev->vf_alloc = nvmsg->msg.v4_msg.vf_assoc.allocated;
|
|
|
|
nvdev->vf_serial = nvmsg->msg.v4_msg.vf_assoc.serial;
|
|
|
|
}
|
|
|
|
|
|
|
|
static inline void netvsc_receive_inband(struct hv_device *hdev,
|
|
|
|
struct netvsc_device *nvdev,
|
|
|
|
struct nvsp_message *nvmsg)
|
|
|
|
{
|
|
|
|
switch (nvmsg->hdr.msg_type) {
|
|
|
|
case NVSP_MSG5_TYPE_SEND_INDIRECTION_TABLE:
|
|
|
|
netvsc_send_table(hdev, nvmsg);
|
|
|
|
break;
|
|
|
|
|
|
|
|
case NVSP_MSG4_TYPE_SEND_VF_ASSOCIATION:
|
|
|
|
netvsc_send_vf(nvdev, nvmsg);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2016-07-06 07:52:46 +08:00
|
|
|
static void netvsc_process_raw_pkt(struct hv_device *device,
|
|
|
|
struct vmbus_channel *channel,
|
|
|
|
struct netvsc_device *net_device,
|
|
|
|
struct net_device *ndev,
|
|
|
|
u64 request_id,
|
|
|
|
struct vmpacket_descriptor *desc)
|
|
|
|
{
|
|
|
|
struct nvsp_message *nvmsg;
|
|
|
|
|
|
|
|
nvmsg = (struct nvsp_message *)((unsigned long)
|
|
|
|
desc + (desc->offset8 << 3));
|
|
|
|
|
|
|
|
switch (desc->type) {
|
|
|
|
case VM_PKT_COMP:
|
|
|
|
netvsc_send_completion(net_device, channel, device, desc);
|
|
|
|
break;
|
|
|
|
|
|
|
|
case VM_PKT_DATA_USING_XFER_PAGES:
|
|
|
|
netvsc_receive(net_device, channel, device, desc);
|
|
|
|
break;
|
|
|
|
|
|
|
|
case VM_PKT_DATA_INBAND:
|
|
|
|
netvsc_receive_inband(device, net_device, nvmsg);
|
|
|
|
break;
|
|
|
|
|
|
|
|
default:
|
|
|
|
netdev_err(ndev, "unhandled packet type %d, tid %llx\n",
|
|
|
|
desc->type, request_id);
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2014-04-22 01:20:28 +08:00
|
|
|
void netvsc_channel_cb(void *context)
|
2009-07-14 06:34:54 +08:00
|
|
|
{
|
2009-09-03 01:33:05 +08:00
|
|
|
int ret;
|
2014-04-22 01:20:28 +08:00
|
|
|
struct vmbus_channel *channel = (struct vmbus_channel *)context;
|
|
|
|
struct hv_device *device;
|
2010-12-11 04:03:54 +08:00
|
|
|
struct netvsc_device *net_device;
|
|
|
|
u32 bytes_recvd;
|
|
|
|
u64 request_id;
|
2009-08-28 07:02:36 +08:00
|
|
|
struct vmpacket_descriptor *desc;
|
2010-04-28 04:23:47 +08:00
|
|
|
unsigned char *buffer;
|
|
|
|
int bufferlen = NETVSC_PACKET_SIZE;
|
2011-09-14 01:59:49 +08:00
|
|
|
struct net_device *ndev;
|
2016-07-06 07:52:46 +08:00
|
|
|
bool need_to_commit = false;
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2014-04-22 01:20:28 +08:00
|
|
|
if (channel->primary_channel != NULL)
|
|
|
|
device = channel->primary_channel->device_obj;
|
|
|
|
else
|
|
|
|
device = channel->device_obj;
|
|
|
|
|
2010-12-11 04:03:55 +08:00
|
|
|
net_device = get_inbound_net_device(device);
|
2011-09-14 01:59:49 +08:00
|
|
|
if (!net_device)
|
2014-02-17 08:38:45 +08:00
|
|
|
return;
|
2016-05-13 19:55:23 +08:00
|
|
|
ndev = hv_get_drvdata(device);
|
2014-04-22 01:20:28 +08:00
|
|
|
buffer = get_per_channel_state(channel);
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-09-03 01:33:05 +08:00
|
|
|
do {
|
2016-07-06 07:52:46 +08:00
|
|
|
desc = get_next_pkt_raw(channel);
|
|
|
|
if (desc != NULL) {
|
|
|
|
netvsc_process_raw_pkt(device,
|
|
|
|
channel,
|
|
|
|
net_device,
|
|
|
|
ndev,
|
|
|
|
desc->trans_id,
|
|
|
|
desc);
|
|
|
|
|
|
|
|
put_pkt_raw(channel, desc);
|
|
|
|
need_to_commit = true;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
if (need_to_commit) {
|
|
|
|
need_to_commit = false;
|
|
|
|
commit_rd_index(channel);
|
|
|
|
}
|
|
|
|
|
2014-04-22 01:20:28 +08:00
|
|
|
ret = vmbus_recvpacket_raw(channel, buffer, bufferlen,
|
2010-12-11 04:03:54 +08:00
|
|
|
&bytes_recvd, &request_id);
|
2009-09-03 01:33:05 +08:00
|
|
|
if (ret == 0) {
|
2010-12-11 04:03:54 +08:00
|
|
|
if (bytes_recvd > 0) {
|
2009-09-03 01:33:05 +08:00
|
|
|
desc = (struct vmpacket_descriptor *)buffer;
|
2016-07-06 07:52:46 +08:00
|
|
|
netvsc_process_raw_pkt(device,
|
|
|
|
channel,
|
|
|
|
net_device,
|
|
|
|
ndev,
|
|
|
|
request_id,
|
|
|
|
desc);
|
|
|
|
|
2009-07-14 06:34:54 +08:00
|
|
|
|
2009-09-03 01:33:05 +08:00
|
|
|
} else {
|
2014-02-17 08:38:45 +08:00
|
|
|
/*
|
|
|
|
* We are done for this pass.
|
|
|
|
*/
|
2009-07-14 06:34:54 +08:00
|
|
|
break;
|
|
|
|
}
|
2014-02-17 08:38:45 +08:00
|
|
|
|
2011-08-26 00:48:59 +08:00
|
|
|
} else if (ret == -ENOBUFS) {
|
2014-02-17 08:38:45 +08:00
|
|
|
if (bufferlen > NETVSC_PACKET_SIZE)
|
|
|
|
kfree(buffer);
|
2009-09-03 01:33:05 +08:00
|
|
|
/* Handle large packet */
|
2010-12-11 04:03:54 +08:00
|
|
|
buffer = kmalloc(bytes_recvd, GFP_ATOMIC);
|
2009-09-03 01:33:05 +08:00
|
|
|
if (buffer == NULL) {
|
2009-07-28 04:47:24 +08:00
|
|
|
/* Try again next time around */
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev,
|
2009-09-03 01:33:05 +08:00
|
|
|
"unable to allocate buffer of size "
|
2011-09-02 03:19:40 +08:00
|
|
|
"(%d)!!\n", bytes_recvd);
|
2009-07-14 06:34:54 +08:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
|
2010-12-11 04:03:54 +08:00
|
|
|
bufferlen = bytes_recvd;
|
2009-07-14 06:34:54 +08:00
|
|
|
}
|
|
|
|
} while (1);
|
|
|
|
|
2014-02-17 08:38:45 +08:00
|
|
|
if (bufferlen > NETVSC_PACKET_SIZE)
|
|
|
|
kfree(buffer);
|
2009-07-14 06:34:54 +08:00
|
|
|
return;
|
|
|
|
}
|
2011-04-22 03:30:40 +08:00
|
|
|
|
2011-04-22 03:30:45 +08:00
|
|
|
/*
|
|
|
|
* netvsc_device_add - Callback when the device belonging to this
|
|
|
|
* driver is added
|
|
|
|
*/
|
2011-05-10 22:54:53 +08:00
|
|
|
int netvsc_device_add(struct hv_device *device, void *additional_info)
|
2011-04-22 03:30:45 +08:00
|
|
|
{
|
2016-05-13 19:55:25 +08:00
|
|
|
int i, ret = 0;
|
2011-05-13 10:35:05 +08:00
|
|
|
int ring_size =
|
|
|
|
((struct netvsc_device_info *)additional_info)->ring_size;
|
2011-04-22 03:30:45 +08:00
|
|
|
struct netvsc_device *net_device;
|
2016-05-13 19:55:25 +08:00
|
|
|
struct net_device *ndev = hv_get_drvdata(device);
|
|
|
|
struct net_device_context *net_device_ctx = netdev_priv(ndev);
|
2011-04-22 03:30:45 +08:00
|
|
|
|
2016-05-13 19:55:25 +08:00
|
|
|
net_device = alloc_net_device();
|
2014-09-04 19:11:23 +08:00
|
|
|
if (!net_device)
|
|
|
|
return -ENOMEM;
|
2011-04-22 03:30:45 +08:00
|
|
|
|
2014-04-22 01:20:28 +08:00
|
|
|
net_device->ring_size = ring_size;
|
|
|
|
|
2011-04-22 03:30:45 +08:00
|
|
|
/* Initialize the NetVSC channel extension */
|
2011-05-10 22:55:41 +08:00
|
|
|
init_completion(&net_device->channel_init_wait);
|
2011-04-22 03:30:45 +08:00
|
|
|
|
2014-04-22 01:20:28 +08:00
|
|
|
set_per_channel_state(device->channel, net_device->cb_buffer);
|
|
|
|
|
2011-04-22 03:30:45 +08:00
|
|
|
/* Open the channel */
|
2011-05-13 10:35:05 +08:00
|
|
|
ret = vmbus_open(device->channel, ring_size * PAGE_SIZE,
|
|
|
|
ring_size * PAGE_SIZE, NULL, 0,
|
2014-04-22 01:20:28 +08:00
|
|
|
netvsc_channel_cb, device->channel);
|
2011-04-22 03:30:45 +08:00
|
|
|
|
|
|
|
if (ret != 0) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev, "unable to open channel: %d\n", ret);
|
2011-04-22 03:30:45 +08:00
|
|
|
goto cleanup;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Channel is opened */
|
2011-09-02 03:19:40 +08:00
|
|
|
pr_info("hv_netvsc channel opened successfully\n");
|
2011-04-22 03:30:45 +08:00
|
|
|
|
2016-05-13 19:55:25 +08:00
|
|
|
/* If we're reopening the device we may have multiple queues, fill the
|
|
|
|
* chn_table with the default channel to use it before subchannels are
|
|
|
|
* opened.
|
|
|
|
*/
|
|
|
|
for (i = 0; i < VRSS_CHANNEL_MAX; i++)
|
|
|
|
net_device->chn_table[i] = device->channel;
|
|
|
|
|
|
|
|
/* Writing nvdev pointer unlocks netvsc_send(), make sure chn_table is
|
|
|
|
* populated.
|
|
|
|
*/
|
|
|
|
wmb();
|
|
|
|
|
|
|
|
net_device_ctx->nvdev = net_device;
|
2014-04-22 01:20:28 +08:00
|
|
|
|
2011-04-22 03:30:45 +08:00
|
|
|
/* Connect with the NetVsp */
|
|
|
|
ret = netvsc_connect_vsp(device);
|
|
|
|
if (ret != 0) {
|
2011-09-02 03:19:41 +08:00
|
|
|
netdev_err(ndev,
|
2011-09-02 03:19:40 +08:00
|
|
|
"unable to connect to NetVSP - %d\n", ret);
|
2011-04-22 03:30:45 +08:00
|
|
|
goto close;
|
|
|
|
}
|
|
|
|
|
|
|
|
return ret;
|
|
|
|
|
|
|
|
close:
|
|
|
|
/* Now, we can close the channel safely */
|
|
|
|
vmbus_close(device->channel);
|
|
|
|
|
|
|
|
cleanup:
|
2014-08-16 03:18:19 +08:00
|
|
|
free_netvsc_device(net_device);
|
2011-04-22 03:30:45 +08:00
|
|
|
|
|
|
|
return ret;
|
|
|
|
}
|