February 16, 2006

Resnet Update Folks

Having only skimmed the update yesterday, today I thought I'd have a good look… Mmmm

Results revealed that delays in response times only occur at times of peak load when the network is congested and we have therefore usefully ruled out the possibility of there being an underlying problem with the handling of DNS.

Which surely reveals an underlying design fault with resnet itself – heck if DNS requests can't get through what chance have my WoW packets got? Seriously…

We are also concerned that the main ResNet firewall is overloaded and are now focussing our attention on this area. As a first step towards reducing the load, we removed the limit on the maximum number of sessions that an individual IP address can set up. This made no difference. We now propose to install an additional firewall to double the capacity.

Translation : we are not sure why resnet is still slow so now we will have a tinker with the firewalls. Two firewalls should be faster than one right?

Seriously though when did the capacity of the firewalls get investigated? Surely firewall logs would be one of the first places to look… oh well.

Aghast with the information so far I almost died when I met the "timetable for change" as I call it:

Thursday 23 February, 9.00am – 10.00am : Implement highest possible priority for the handling of DNS traffic on all components at the core of the network.

Now surely the network is already setup to treat DNS requests as high priority? Maybe not…

I must admit since the very start of the year I have been very vocal to my fellow students about ResNet, but never actually gone along to any of the focus meetings. That will change as I have read only 2 people turned up to the last few…

Ok I admit that finally we are moving somewhere with Resnet but I mean come on, 4 months to get to this point is a frankly a big fat joke which is further compounded by the lack of any proper compensation.

Does anyone have a detailed Resnet network topography to hand?


- 41 comments by 5 or more people Not publicly viewable

[Skip to the latest comment]
  1. Mathew Mannion

    Maybe ResNet would be much faster if they banned WoW.

    16 Feb 2006, 13:31

  2. Matthew Jones

    Maybe the world would be much better if they banned WoW.

    16 Feb 2006, 13:34

  3. Leigh Robinson

    Maybe the world would be much better if they banned WoW.

    You may actually be onto something there – it is a huge time sink… :D

    Maybe ResNet would be much faster if they banned WoW.

    This though is just rubbish and you know it! :P

    16 Feb 2006, 13:45

  4. Mathew Mannion

    I like the way that all the so called student "experts" talk about ResNet, when they seem to forget the fact that when so many of them use P2P and online gaming and so forth that at peak times, 80mbit only goes into 8000 on-campus students so many times.

    16 Feb 2006, 14:25

  5. Leigh Robinson

    I like the way that all the so called student "experts" talk about ResNet, when they seem to forget the fact that when so many of them use P2P and online gaming and so forth that at peak times, 80mbit only goes into 8000 on-campus students so many times.

    Yes yes it does. But when you block properly all external interfaces from such traffic and the network is still pathetic what do you then blame it on? That most of the students know how circumvent the firewall via other proxies, tunnelling, etc…

    No, all you can conclude is there is a fundamental problem irrespective of P2P activity. You can strike online gaming from your comment too – they use relatively little bandwidth, they rely on low latentcy.

    Have you even noticed that the traffic shaper is not on the external interface? We should have done but I (we?) have not even noticed…

    16 Feb 2006, 14:39

  6. The problem is patently not P2P (which has always been controlled) and Online gaming (which Warwick nerfed towards the end of my first year). The majority of file-sharing goes on within the network and is not fetched from outside the network (at least that used to be the case).

    The Uni's primary worry isn't in hitting the bandwidth limit and therefore bringing anything else to standstill… but the cost of internet-usage. The Uni has, therefore, fiddled about with the network to reduce costs to such an extent that they're not sure which pipe they've broken is the one that's leaking.

    And no, I'm not an expert. It doesn't take an expert to understand the basics of network trafficking… I think you were unnecessarily insulting towards some of our resident geeks too… I've known plenty of "Experts" who could actually outdo most of the experts working on the network – just go to your local Compsoc Lan party. Heck looking at your profile you should know some of these people – eg. Silent Dave? Is he still around?

    16 Feb 2006, 14:53

  7. Mathew Mannion

    How many of those "experts" have actually worked on ResNet? It's blind idiocy from the majority of people who claim to know the ins and outs of ResNet and know exactly what's wrong at every point. If you think you know plenty of people in CompSoc who can outdo experts who spend all day every day working on these things, then you have a very naive view on things. I know plenty of people in CompSoc who think they know more about networking than the people in ITS, but how many of them have worked on a large multi-environment network? I'm betting none.

    I'm just tired of seeing posts from people who have absolutely no idea about these things really. It's fair enough to complain, but when it gets to the point where people are claiming to have more knowledge than the people who get paid to do this, it's rather ridiculous.

    16 Feb 2006, 15:08

  8. A LAN party does not compare to a network with many thousands of network ports and hundreds of pieces of network equipment. That's like saying that someone who knows how to balance their personal finances must surely also be an expert at multinational corporate finance.

    16 Feb 2006, 15:43

  9. A LAN party does not compare to a network with many thousands of network ports and hundreds of pieces of network equipment. That's like saying that someone who knows how to balance their personal finances must surely also be an expert at multinational corporate finance.

    I think you misunderstood me. Attend the next Compsoc LAN to meet the people from Compsoc, rather than use it as an example of their expertise.

    I know plenty of people in CompSoc who think they know more about networking than the people in ITS, but how many of them have worked on a large multi-environment network?

    I know a few who have gone on to work on large, multi-environment networks… I'm not sure you know LHO/Jago so it's pointless to drop their names :P. To paraphrase a great quotation from Harold and Kumar "Just because I'm hung like a horse, doesn't mean I have to do porn." A lot of the people who are, no doubt, more talented than many who are working on networks simply won't go and work on networks because it's either too boring or they'd like to do something else.

    And I think your argument is somewhat lazy. It presupposes that anyone who isn't actually an expert in something through institutionalised education cannot dare to comment (cynically so) on issues to do with the area of expertise. I don't have to be an absolute expert on statutory interpretation to have an opinion on statutes and to believe that someone is wrong or deliberately misinterpreting a statute…

    It's fair enough to complain, but when it gets to the point where people are claiming to have more knowledge than the people who get paid to do this, it's rather ridiculous.

    Let's leave the "who knows more" crap aside for the moment… I don't believe that the actual poster of this blogged claimed to know more than ITS or Piazza whoever's dealing with things these days… but rather cynically interpretted what they were saying… In fact at most he didn't believe their lame excuses. It does not mean that he will want to wade in proclaiming "I can do your job far better than you can".

    16 Feb 2006, 15:54

  10. Oh yes and it goes back to what I said before:

    And no, I'm not an expert. It doesn't take an expert to understand the basics of network trafficking
    …

    And this really is the case. I don't have to be an expert on linguistics to know that someone telling me that a verb is declined is blowing smoke up my ass.

    16 Feb 2006, 15:57

  11. Leigh Robinson

    A LAN party does not compare to a network with many thousands of network ports and hundreds of pieces of network equipment. That's like saying that someone who knows how to balance their personal finances must surely also be an expert at multinational corporate finance.

    True of course. Though I would argue that the difficulty in setting up a large network does not scale up in the same way as your personal finance analogy.

    Kieran I have to ask since I know your technical – do you think that the state of the Resnet is due to mis-management or abusive users? What network of this size does not have a bulletproof packet filtering appliance? DNS should NOT take 10 seconds no matter the network load. Simple.

    16 Feb 2006, 15:58

  12. Leigh Robinson

    I'm just tired of seeing posts from people who have absolutely no idea about these things really. It's fair enough to complain, but when it gets to the point where people are claiming to have more knowledge than the people who get paid to do this, it's rather ridiculous.

    I see your point. But when they can demonstrate they are "experts" then I will be convinced. I actually contacted the desk last term asking for a technical reason why I could not access any DNS servers – they could not give me one because they did not know. PLEASE - after hours of searching they still did not know? How many things can cause a total loss of DNS? Why isn't the DNS system as resilient as it should? I could go on but wont as I hope the worst are far behind us…

    16 Feb 2006, 16:01

  13. Leigh Robinson

    Furthermore I am not saying in any way I could do better… It is obviously easy to look at their actions and criticise – I just wonder if there is a lot more than meets the eye, specifically cost wise. I am sure that talented network engineers would NOT take 4–5 months to come back and say "well double the firewalls and prioritse DNS and your set…"

    Crap.

    16 Feb 2006, 16:04

  14. Unfortunately I can not comment on what is going on with ResNet because ITS is a large department is it really isn't my area of expertise. However, I can say that everyone involved is working hard to fix these problems. Their jobs are made all that much harder as it seems to be a bit of an arms race between the users and ITS.

    16 Feb 2006, 16:09

  15. Leigh Robinson

    Their jobs are made all that much harder as it seems to be a bit of an arms race between the users and ITS.

    How so? Am I missing something?
    I hope you dont mean all the complaints and criticism levied at them…

    16 Feb 2006, 16:13

  16. No no, I just mean that as ITS block stuff, users try to circumvent those blocks.

    That most of the students know how circumvent the firewall via other proxies, tunnelling, etc
    …

    Anyway, I'm not an expert in this area, just sticking up for my colleagues :)

    16 Feb 2006, 16:17

  17. Leigh Robinson

    No no, I just mean that as ITS block stuff, users try to circumvent those blocks.

    You can't seriously believe the percentage of users with sufficient know-how are responsible for all the performance issues!?

    Anyway, I'm not an expert in this area, just sticking up for my colleagues :)

    :D

    16 Feb 2006, 16:20

  18. I blame resnet for me not having a level 60 yet :/

    16 Feb 2006, 17:05

  19. I blame resnet for me not having a level 60 yet :/

    UBRS FTW

    16 Feb 2006, 17:19

  20. Chris May

    The Uni's primary worry isn't in hitting the bandwidth limit and therefore bringing anything else to standstill, but the cost of internet-usage.

    I presume you're not talking about cost in financial terms: You do know, don't you, that the uni doesn't pay by the byte for it's JANET connection? It's a flat charge for all the traffic you can eat, up to whatever the width of the pipe is (I think it was 4GB/s the last time I asked but that was a couple of years back)

    Like Kieran, I don't know enough about resnet to know what the source of the problems is, but I do know that they've got some very smart and dedicated guys looking at it. So I can only conclude that it's something difficult and complicated.

    16 Feb 2006, 18:07

  21. Leigh Robinson

    I presume you're not talking about cost in financial terms: You do know, don't you, that the uni doesn't pay by the byte for it's JANET connection? It's a flat charge for all the traffic you can eat, up to whatever the width of the pipe is (I think it was 4GB/s the last time I asked but that was a couple of years back)

    I did not mean the charge from the external interface. I mean the costs to actually bring the network upto speed – by replacing the core network gear. If it truely is the load that is causing DNS to take upto 10 seconds then it needs to be ripped out and replaced with newer equipment. I doubt the smartest network bods will be able to eek a decent performance out of the system if it is truely load.

    Like Kieran, I don't know enough about resnet to know what the source of the problems is, but I do know that they've got some very smart and dedicated guys looking at it. So I can only conclude that it's something difficult and complicated

    The trouble is that from my experience it cannot be that hard. It just can't. I have experience of BT's internal WAN/LAN and that hosted around 35,000 PC's – simply as a contrast this shows that it is entirely possible to have a network this large. To my mind it just comes down to how much are you prepared to pay to get it running at a given level of performance…

    Really I am not trying to be funny here or claim the engineers themselves are incompetent in any way. If they can read this maybe they could stick a technical explaination of the current problem as a reply?
    Please!

    16 Feb 2006, 18:23

  22. Skunky

    Surely that's a ridiculous network architecture that routes DNS requests bound for a Campus network based resolver through an interface intended for external network traffic that has been overloaded for years now? The fact that there have been 58 updates to the warwick.ac.uk zone today alone makes me wonder too…

    Incidently, Warwick doesn't pay anything for its connection to MidMAN as it's all paid for by the JISC.

    16 Feb 2006, 18:52

  23. Leigh Robinson

    Surely that's a ridiculous network architecture that routes DNS requests bound for a Campus network based resolver through an interface intended for external network traffic that has been overloaded for years now?

    It doesn't the DNS servers are 192.146.136.13–14

    Yet the DNS is still slow… Anybody know how the DNS is configured? do those ip's point to single machines or are they virtual ip's interfacing with a load balancer and a cluster of DNS servers behind?

    16 Feb 2006, 19:09

  24. I mean the costs to actually bring the network upto speed – by replacing the core network gear.

    The core can easily handle the loads placed on it, and all the edge is switched at gigabit up to distribution (or was, last time I had anything to do with it).

    I suspect that the problem was with the traffic flow in and through DMZ.

    I have experience of BT's internal WAN/LAN and that hosted around 35,000 PC's – simply as a contrast this shows that it is entirely possible to have a network this large.

    And doing most of the things that resnet's users do on a BT network would be a sacking offence. There is absolutely nothing similar between a large corporate network and a university residential network.

    But hey, I'm nothing to do with it. fwiw I believe that resnet should either enforce its policy properly (discipline people doing filesharing) or let people run DC++ hubs on campus. More on-campus filesharing is less off-campus filesharing. We got by fine with a 40Mbps pipe when nobody needed to use it because it was all available on campus…

    16 Feb 2006, 19:10

  25. skunky

    The DNS passing through the 'resnet's external interface is the suggestion of the parent poster.

    With regard to letting ITS off the hook because Warwick Piazza is a university campus – The University of Manchester manages to transfer about 4x as much traffic quite happily.

    I'd disagree that we got by fine on a 40Mbit pipe. Were you on campus last year?

    16 Feb 2006, 19:18

  26. Skunky, no – 3 years ago. But P2P was still everywhere, the difference was that everybody used an on-campus DC++ hub.

    16 Feb 2006, 19:25

  27. Chris May

    Manchester's an interesting example. Here's a snippet from their AUP:

    link

    " Disconnection policy: any machine which is disconnected due to misuse of the network facilities including, but not limited to, those listed above will be regarded as a serious disciplinary matter within the Halls of Residence and the University. Misuse will result in the immediate termination of your access to the Hornet Service. It may also result in disciplinary action being taken by the University or the British police. Any machine which is disconnected and is authorised to be reconnected to the Hornet Service will be liable for the following fees.

    (i) Viruses: £35 administration fee for reconnection after the third virus disconnection. No disconnection period is defined as users can seek reconnection once the virus threat is removed.

    (ii) General Abuse: £50 reconnection fee to be paid after a 28 day disconnection period.

    (iii) Copyright Abuse: £70 reconnection fee to be paid after a 56 day disconnection period.

    (iv) Network Abuse: £100 reconnection fee to be paid after a 56 day disconnection period.
    "

    I suspect that if ITS at warwick was allowed to implement something as draconian as this, then managing resnet would be a whole lot easier.

    16 Feb 2006, 19:29

  28. Leigh Robinson

    I suspect that the problem was with the traffic flow in and through DMZ.

    How is it currently configured? Is it with multiple firewalls or a 3-node topography? – im genuinely curious… :P

    We are also concerned that the main ResNet firewall is overloaded and are now focussing our attention on this area.

    Some decent balancing through the DMZ is needed then. Stick some F5's in there – they seem to work wonders….

    16 Feb 2006, 19:34

  29. Skunky

    Max,
    Even with the dc++ hub still present last year, the network was still overloaded to the point of web browsing timing out at peak times. I remember several times when other people (not me, I'm pure as the driven snow) would resort to doing little tricks with department SSH servers to get round the flapping resnet firewall.

    Chris,
    Draconian action? According to the resnet AUP, Warwick go straight for the £100 fine! That's possible for any offence, not even anything as serious as 'Network Abuse'. As for disconnections, Warwick have certainly dealt those out in the past.

    16 Feb 2006, 19:42

  30. Last year i lived in Lakeside and enjoyed a flawless internet connection that is to say 95% of requests regardless of protocol were successful. This year i live in Claycroft and the percentage of successful requests again regardless of protocol it’s more like 70%. It does not take a degree in computer science to know that clearly some variable associated with Resnet has changed. What i would like to know is which one? I mean the number of users has not gone up exponentially has it? What about bandwidth? Have we sold some of our bandwidth to NTL perhaps?

    I honestly believe the engineers managing Resnet are fully qualified and capable of doing the job therefore i am sure they are doing the best they can with what they have or perhaps what they have left after senior management made cutbacks maybe. Also remember that it’s not the engineers you talk to when you call ITS, its just some poor call centre person that fields your call and who has a script with a list of pre written answers.

    17 Feb 2006, 00:49

  31. While I'm certainly not an expert on the subject (I'm not even a computer scientist) I do know a little about traffic shaping and networks and unlike most of the others I have been to almost every focus group meeting in an attempt to actully help ITS sort out the problems.

    The problem, as much as people refuse to believe it, is P2P traffic. ITS maintain that all the systems in their AUP are to blame and here I do have to disagree with them. I believe that BitTorrent is almost entirely the problem. It is a system desgined extremely well by a bunch of very clever people in order to use as much bandwidth and connections as possible, and it does this very well. The now large number of people being banned and fined have been using these system (espescially BitTorrent) as the traffic signature of these stand out a mile on the logs. Yes, they do have traffic logs and if you came to the meetings you'd be able to see them.

    The reason it wasn't so much of a problem before is simply that the use of BitTorrent was much less. The on campus hubs for DC++ obviously alleviated much of the problem and ITS really did shoot themselves in the foot when they took them all down this year. This was however, a policy of the university management who I don't think even vaguely understood what it would mean.

    It should also be pointed out that ResNet is a very large but most notably very non-standard system. It was installed by an external provider who were relieved of their control of the network very early this academic year. It seems to me that they installed a stock system desgined for X number of users, and probably for a buisness environment. Unfortunately ResNet is very different. As students our usage of it is both a lot heavier and a lot more complicated and the equipment is place was not up to the job.

    My final point is that the hours of usage of ResNet are very different to office hours. The ITS team (should) finish work at about 5 and the peak times of ResNet are around 6pm-11pm and on weekends. It might be an idea for people (and especially the student's union) stopped worrying about getting a few pounds a week compensation for the point of it and perhaps lobbied the University and Warwick Accomodation to provide ITS with a staff member resonsible for only resnet, at the times resnet operates at its peak.

    (Apologies if I've covered anything again, I didn't have time to read all the comments)

    17 Feb 2006, 09:56

  32. If you think you know plenty of people in CompSoc who can outdo experts who spend all day every day working on these things, then you have a very naive view on things. I know plenty of people in CompSoc who think they know more about networking than the people in ITS, but how many of them have worked on a large multi-environment network? I'm betting none.

    Mmm, true. But then, the CompSoc techteam are rarely the people suggesting that ITS suck, in my experience. I'd love the chance to have a look at ITS's firewall rules, but it's a hefty responsibility, and a difficult job, I'd imagine.

    Attend the next Compsoc LAN to meet the people from Compsoc, rather than use it as an example of their expertise.

    Signups are open for members, and guest signups open on Wednesday. ;)

    17 Feb 2006, 12:58

  33. "My final point is that the hours of usage of ResNet are very different to office hours. The ITS team (should) finish work at about 5 and the peak times of ResNet are around 6pm-11pm and on weekends. It might be an idea for people (and especially the student's union) stopped worrying about getting a few pounds a week compensation for the point of it and perhaps lobbied the University and Warwick Accomodation to provide ITS with a staff member resonsible for only resnet, at the times resnet operates at its peak."
    I'm not on campus but if I was this year, I wouldn't care about the technical stuff, I wouldn't care about how ITS allocate it's staff and what hours they work – it's really quite simple:
    Those on campus pay for a service, this service should either be made to work, or it's users refunded. I don't care how they do, I don't care what problems they have – if it were out of the goodness of thier hearts they were doing this, I would. But it's a commercial service, and they're failing to provide the service they've promised.
    This is some bizzare disconnect (no pun intended) that seems to exist purely in the IT industry – NTL and BT often have simmilar service problems, PC games and applications are sold to users for £30 and then don't work properly and need patches (which sometimes never arrive) and we put up with it because "ah, it's computers innit. complicated things".
    You pay for something, you should get it. If I only turn up for 80% of my exams the univeristy isn't going to turn around and say 'well you were there most of the time so there's no reason not to give you a full degree'.
    If they can't fix it, hire people that can. If the technology is the problem, replace it. If it's going to take a long time, hire more people. If necessary, charge more for the service – but don't sell us something that doesn't work.

    18 Feb 2006, 20:08

  34. The worst part is that not only do i pay for the internet, we give money to the "union" (who keep assuring us they do stuff with it), and they are doing a far worse job and arent even trying to do any better. what sort of union gives up campaining for compensation because it doesnt want to annoy anyone! i'm pissed off enough as it is but if i was one of the people recieving the princely sum of £8 for a total loss of internet for *2 WEEKS* then i'd be more than a little peeved. students cant do much for themselves because of the costs involved, so that should be where the union comes in. It's there in black and white for them in the contract, and i tested my internet earlier and clocked the quite amazing speeds of 2x dial up, or up to 20 times crappier than home broadband. one test actually failed to load the results page. To make matters worse we voted for them (the union) to do something and they still do nothing, so we have neither a union nor a democracy, nor web access… lovely… anyway, at least ITS sorted out my HL2 ping; 400 was just not playable. :-)

    getting back on topic, does anyone else get dead time every so often (every 10 minutes or so) where web pages (external) refuse to load at all for maybe 30 seconds, then all of a sudden pages load fairly quickly, all the while msn and whatnot "functioning" normally? reminds me of when my home router was on the blink and would continually restart, except this is clearly more specific than a router with it just seeminly affecting web pages. They seem to be messing around swapping all this hardware about… i sure hope it's not a quirky cable somewhere…

    19 Feb 2006, 03:56

  35. I don't know what all you guys are complaining about, it's impossible to get a room on campus in order to attain the lousy internet connection in the first place. However it must be said that "ResNet" (whatever happened to Warwick Piazza, sniff) worked almost perfectly in my first year (2 years ago) and since they messed about with it after that it's gone to hell therefore my theory goes, undo everything you've done in the last 2 years and it just might fix it, an Edit::Undo if you will…

    19 Feb 2006, 12:25

  36. This is a page from the website of a company called 'Data Integration'. It makes interesting reading:
    http://www.dataintegration.com/casestudy_universityofwarwick.asp

    19 Feb 2006, 22:49

  37. Leigh Robinson

    It should also be pointed out that ResNet is a very large but most notably very non-standard system. It was installed by an external provider who were relieved of their control of the network very early this academic year. It seems to me that they installed a stock system desgined for X number of users, and probably for a buisness environment. Unfortunately ResNet is very different. As students our usage of it is both a lot heavier and a lot more complicated and the equipment is place was not up to the job.

    Then equipment needs to be upgraded not tinkered with to eek performance. If this is the true problems then as I stated earlier it is more down to cost and what they can recoup from students to pay for it. Perhaps Warwick should turn down some of the modern art tosh that litters the campus and get some donations for a sponsored netowork.

    My final point is that the hours of usage of ResNet are very different to office hours. The ITS team (should) finish work at about 5 and the peak times of ResNet are around 6pm-11pm and on weekends. It might be an idea for people (and especially the student's union) stopped worrying about getting a few pounds a week compensation for the point of it and perhaps lobbied the University and Warwick Accomodation to provide ITS with a staff member resonsible for only resnet, at the times resnet operates at its peak.

    Again Im sorry to say but too bad for them. Get people in to cover the evenings, simple as that – increase the costs to us if you have to, I want a service that works.

    20 Feb 2006, 02:39

  38. Leigh Robinson

    Oliver that site is quite informative.

    It is clear to me that the internal ethernet structure is more than upto the job and the problem is definitely with the external interface and as suggested the traffic that is through the DMZ.

    20 Feb 2006, 02:43

  39. skunky

    Was there ever any doubt of that?

    Using departmental proxy servers to avoid the piazza's flapping interfaces, 'people' could regularly get 50Mbps transfer rates to other sites on JANET.

    20 Feb 2006, 23:20

  40. Leigh Robinson

    Of course. Though my point was that it has taken then 4 months to even start to (officially) look at the firewalls and the DMZ

    Also, if anyone has any details of the next focus meeting could you post them in here? I'll have another search on the ITS site for them but my quick look last night turned up goose eggs.

    20 Feb 2006, 23:48

  41. With regard to the comment that the ITS team finish at 5pm, this is not strictly true. The helpdesk is open until 8pm monday – friday and 10–5 saturday and sunday.

    23 Feb 2006, 15:26


Add a comment

You are not allowed to comment on this entry as it has restricted commenting permissions.

February 2006

Mo Tu We Th Fr Sa Su
|  Today  | Mar
      1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28               

Search this blog

Tags

Galleries

Most recent comments

  • Try the otenet.gr (greek isp) I downloaded a version there that worked perfect on Vista. The origina… by terry on this entry
  • If you are a tiscali customer they have an updated "installation disc" which uyou download and burn … by pothead on this entry
  • Hi all, Anyone still reading this should take note of my "Final Final Edit" paragraph. It seems the … by Leigh Robinson on this entry
  • IT'S NOT THAT ATHIESTS DON'T BELIEVE IN GOD,IT'S JUST THAT CHRISTIANS DON'T BELIEVE IN ATHIEST.YES.I… by JD on this entry
  • Hi all, could someone be so kind and send me a link or directly the driver 2.2.10.20 for my sagem 80… by markus on this entry

Blog archive

Loading…
Not signed in
Sign in

Powered by BlogBuilder
© MMXIX