Skip Ribbon Commands
Skip to main content

Quick Launch

Home
 

 Posts

 
April 05
Sonicwall NSA 2400 Hangs with Heavy Replication Traffic on a Layer 2 link

We're starting to implement more Layer 2 connections from Cable Providers for high speed WAN links.  The prices of these links are very reasonable compared to MPLS and Frame Relay.  However it's very important to properly throttle and match the same speed on the firewall as the Layer 2 connection.  We typically connect the Layer 2 connection to a separate interface on the firewall so we can control the traffic between the LAN and WAN and prevent unnecessary traffic on the WAN link.

For example if you’ve purchased a 50mb/sec Layer 2 WAN link between two sites the Cable provider will typically provide a 100mb/sec Ethernet connection at each site.  If you don’t properly throttle the WAN link on the firewall it will try to push 100mb/sec across the Layer 2 network.  Since the Layer 2 connection can only handle 50mb/sec you will drop packets on the WAN link.  In a worse case scenario, we’ve seen the firewall/router hang or the interface on the firewall randomaly disconnect without any entries in the firewall logs.
To limit the bandwidth on a Sonicwall NSA 2400 complete the following steps:
  1. Login to the Sonicwall NSA 2400 with a Web Browser.
  2. Click on Firewall Settings.
  3. Click on BWM ( Bandwidth Management).
  4. Set the Bandwidth Management Type to Global.
  5. Click on Network and edit the interface of the Layer 2 WAN link.
  6. Click on the Advanced Tab.
  7. Check the Enable Egress Bandwidth Management checkbox and set it to the speed of the Layer 2 WAN speed.  For example if the connection is 50mb/sec you would enter 50000.
  8. Check the Enable Ingress Bandwidth Management checkbox and set it to the speed of the Layer 2 WAN speed.  For example if the connection is 50mb/sec you would enter 50000.
  9. Click Ok.

Hopefully this post may prevent a lot of grief when utilizing these types of WAN links.

 

January 15
SharePoint 2010 Search is broken and you have difficulty saving attachments on SharePoint

A recent Windows Update File KB2756920 causes multiple problems on SharePoint 2010 running on Windows Server 2008 R2. If you install this patch on your SharePoint Search Server it will break SharePoint Search. You will receive an Internal server error exception error whenever you try to search on SharePoint 2010. Additionally, we noticed problems when retrieving and/or saving attachments on the portal.  In SharePoint Central Administration you may receive a Health Analyzer warning that the Security Token Service is not available.

 

Fortunately, the fix is simple.  Uninstall the patch KB2756920, reboot the server and verify that search and file uploads and download are now working.  You can install a newer hotfix for 3.5 sp1 KB2637518 or install Windows Server 2008 R2 Sp1, which will not break search.

 

July 26
AT&T DNS Server is not working.

We recently had a client install a new AT&T Fiber line for Internet access.  After the line was installed, it didn't work properly.  AT&T did some troubleshooting and said the line was OK.  I performed some troubleshooting and noticed that one of the DNS servers 12.127.17.71 that AT&T provided was randomly timing out during a simple ping test. 

I changed the DNS fowarders on the client's DNS servers to  12.127.16.67 (AT&T), 198.6.1.5 (uu.net) and 198.6.1.6 (uu.net).  After this change the Internet connection was fast and stable. 

If your environment is configured to use 12.127.17.71 as a DNS Server and you're having problems reaching sites on the Internet, consider removing this DNS server from your configuration and use a different ISP's DNS server.

December 22
Secure Remote Access with Sonicwall SSL VPN and PocketCloud on iPAD

Sonicwall recently released an iPAD client for their newer Secure Remote Access (SRA) SSL VPN Appliances, NSA and E Class firewalls called Sonicwall Mobile Connect.  It's a free download available from the App Store at ​http://itunes.apple.com/us/app/sonicwall-mobile-connect/id466931806?mt=8.  Setup is really simple and works well with the SRA1200.  For remote desktop access you can download an iPAD RDP Client like Wyse PocketCloudPro http://itunes.apple.com/us/app/wyse-pocketcloud-pro-remote/id326512817?mt=8 to gain access to a remote desktop.

Great solution if you want to travel light!

October 04
Exchange 2010 Service Authentication Fails with UnexceptedExchangeAuthBlob

I recently solved a very nasty authentication problem with two Exchange Servers running as VMs on two separate ESX hosts.  Randomly the servers would receive an error in the Application Event Viewer ID 1035 Inbound authentication failed with error UnexpectedExchangeAuthBlob for the Receive connector.  This error would occur when the Exchange Servers tried to send mail to each other.  When this error occurred, mail would back up in the mail queues.  Sometimes the Exchange Servers would fix themselves and sometimes I had to manually restart the Exchange Transport Services on both servers to get the mail flowing again.

Because the errors occurred randomly, the problem was very difficult to troubleshoot.  Sometimes mail would work for a few days, and sometimes it would break once or twice a day.  I also noticed I was receiving a lot of time adjustment notifications in the System Event Logs.  Sometimes the time change adjustment was minimal, but randomly time was adjusted by roughly six minutes.  There was a correlation between 1035 errors in the Application Event Log and the six minute time adjustments in the System Event Log.
 
Time synchronization was properly configured according to Microsoft’s best practices by synchronizing time of the PDC Emulator to an external NTP time source and then all domain members synchronizing their time to the PDC Emulator.  This appeared to work fine.  In the VMware Tools, both VMs were NOT set to synchronize their time with the ESX host.  Obviously the VMs were getting their time from somewhere other than the PDC Emulator, but where?
 
One of my co-workers Kris Kroll suggested that I look at the ESX hosts.  Bingo!  Even through both ESX hosts were configured to synchronize their time to an external NTP server, they were roughly six minutes off from the time on the PDC Emulator.  As soon as I adjusted the time on the ESX hosts, the problem went away.  Even when you don’t’ have the checkbox checked in VMware tools to NOT synchronize time of the VM with the ESX host, it can still receive time updates from the ESX host.  Some of these times are pretty obvious like during a reboot, or server startup.  Because I finally narrowed down the authentication problem to be time related, I ran the command:
 
W32tm /resync /rediscover
 
Evidently when you run this command a Windows Server has to get approximately five good time synchronizations before it “calms” down and doesn’t try to synchronize time repeatedly.  Because time was getting adjusted by roughly six minutes during this period, the VMs never calmed down and repeatedly tried to synchronize their time over and over.  This caused the Exchange Servers to randomly have Authentication Problems because their clock would skew by more than five minutes causing Kerberos Authentication errors. 
 
Bottom line – ESX Host time does matter, even when it’s not supposed to matter.  Hopefully this blog post will help you solve any Kerberos authentication issues with VMs running on ESX. 

 

September 15
FastSCP Times out when copying large files to an ESX host

I had to restore some *.vmdk files to an ESX host and I used FastSCP to perform the restore.  During the copy I received a error message that connection timed out at the host.  I performed the regular troubleshooting steps and made sure that the proper ports on the firewall were opened on the ESX ​host, but the copy still continued to fail.

There were some VMs that were sharing the same NIC that was used to perform the copy to the ESX host with FastSCP.  I moved all of these VMs to a different NIC on the host and was able to successfully complete the copy.  If you're having difficultly restoring files to an ESX host, try using a dedicated NIC in the host for your FastSCP restores.  I got roughly triple the copy speed with FastSCP compared the WinSCP.

July 25
Verizon Outage in Southern California

It appears that Verizon has a major outage in the Southern California Area today.  According to Tweets on Twitter, Wireless, DSL and FIOS services may be affected.  In Arcadia 911 service is down on the Verizon wireless network.

April 24
Nextiva Phone System and Sonicwall NSA 2400

We have a client that recently installed a Nextiva Phone System http://www.nextiva.com/.  The Nextiva  phone system uses Cisco 303G SIP Phones that connect the Nextiva's phone switch in the cloud.  When we contacted Nextiva for help on configuring their phone system with the Sonicwall NSA 2400 their support indicated that their phone system is NOT compatible with Sonicwall firewalls.  They do ship a Linksys Wireless router that does work with their phone system, but we wanted to use the Sonicwall to leverage the enhanced security featuress and automatic failover to a backup Internet line.  We found a combination that works with the Nextiva phone system and the Sonicwall NSA 2400.  Here are details:

  1. Sonicwall NSA 2400 with the firmware 5.6.2.1-4o.  This SIP enhanced firmware is specifically designed to work with VoIP phones.  This firmware version is availabe on mysonicwall.com.  Make sure that you use this version of the firmware, the latest version of the firmware will cause problems with dropped calls.
  2. VoIP Settings.  Under the VoIP settings, make sure the following items are checked:
    1. Enable consistent NAT.
    2. Enable SIP Transformations.
    3. All other items should be left unchecked.
  3. Outbound Firewall rule.  If you're limiting outbound network traffic from the LAN to the WAN, create a rule to allows UDP traffic on ports 5060-5070 to the IP address of the Nextiva phone sytem.

Before making these changes we received complaints about dropped calls and difficulty dialing out.  But, after the firmware was changed and we made these changes, the phone system has been very stable.  Hopefully this post will help you get a Nextiva phone system up and running with Sonicwall firewall.

March 23
Internet goes down every six hours with Sonicwall NSA 2400 and Verizon FIOS Line

If you have a Sonicwall NSA 2400 or TZ150 (this may effect other Sonicwall models as well) and a Verizon FIOS line, you may find that the Internet goes down every six hours or so and you have to reboot the firewall to restore Internet access.  This can happen if your Sonicwall is configured with more than one external IP address and because of the way FIOS handles it's ARP broadcasts.  Here are the possible workarounds:

 

  1. Configure the Sonicwall with a single static IP address and use NAT Port Service remapping to share a single external IP address and complete the ARP changes in item 3.3 below.
  2. OR
  3. Add in a static route that includes the unused external IP addresses on the firewall.
    1. FIOS IP Block Network Object.  Under Network, Address Objects, create an Address Object range that includes your usable static block, but excludes the IP address of your Sonicwall WAN Interface.
      1. Name:  FIOSStaticIPs
      2. Zone Assignment:  WAN
      3. Type: Range
      4. Starting IP Address: <FIOS IP Block Start Range excluding your WAN IP Address>
      5. Ending IP Address: <FIOS IP Block End Range>
    2. Static Route.  Under Network, Routing, add in a route policy for the network object you just created.
      1. Source: Any
      2. Destination:  FIOSStaticIPs
      3. Service: Any
      4. Gateway: 0.0.0.0
      5. Interface <FIOS WAN Interface usually X1>
      6. Metric: 20
    3. Change the ARP Settings on the Sonicwall.
      1. Go to <ip adress of the sonicwall>/diag.html and click on Internal Settings.
      2. Under the ARP Settings check the following boxes:
        1. Enable ARP bridging
        2. Enable open ARP behavior (WARNING: Insecure!!)
        3. Limit ARPS of non-responsive IPs.

After you add this static route and modify the ARP settings, you should be able to use all of the external IP addresses in your static block and not have the firewall go down every six hours.  Changing these ARP settings on the Sonicwall will make you more vulnerable to an ARP Poisoning attack.  Your static block range must exclude any IP address that are assigned to physical interfaces.  The Internet will go down without the Static route and ARP changes because of the way the FIOS network is built.  The Verizon network sends ARP requests from 0.0.0.0 (an unknown network) and the Sonicwall drops these packets because it thinks it's an ARP cache poisoning attempt.  Because the Sonicwall does not reply to these requests, Verizon FIOS thinks nothing is connected to the static block and drops the connection after six hours.

March 10
vSphere 4.1 with SAS Tape Drives

Although not officially supported by VMware, we have discovered a configuration that works with SAS Tape Drives that are directly connected to an ESX 4.1 Host.  VMware only officially supports U320 Tape Drives, not the newer SAS tape drives.  Unfortunately the newer tape drives like the HP LTO5 drive are only available with the newer SAS or Fibre Channel interfaces.  Here are the details:

  • HP SC44Ge Host Bus Adapter Card Part Number 416096-B21.  This is the only card we know that works with a SAS Tape drive.
  • If you're using an external drive, you must purchase a SAS cable that has a SFF-8470 (computer) and SFF-8088 (tape drive). 
  • HP External LTO5 drive.
  • ESX 4.1 with Update 1.
  • Backup Exec 2010 with the RALUS Agent.
  • Run the following command on the ESX Console:

     

esxcli nmp satp deleterule --satp VMW_SATP_ALUA --claim-option tpgs_on

vSphere 4.1 changed the way tape drives are handled, so you must run this command in order for the VM to see the tape drive.

Using this configuration you should be able to add the SCSI tape drive to a VM running on the ESX host. 

 

1 - 10Next
 

 

 
 

 ADS Consulting Group Links

 
  
ADS Consulting Group Website
ADS Consulting Group's Better Business Bureau Rating
Alan Sugano discusses storage on RunAs Radio
Alan Sugano's Real World Troubleshooting Manual Book on Amazon
Alan Sugano's Tweets
Alan Sugano's Whitepaper on ESX Enterprise Backup and Recovery
Alan Sugano's Windows IT Pro Articles
Google Search for Alan Sugano
SQL Server 2005 Complete Training DVD's by Blast Through Learning
WinConnections Conference Site