Message boards :
Number crunching :
What is so hard about Linux?
Message board moderation
Previous · 1 · 2 · 3 · 4 · 5 · Next
| Author | Message |
|---|---|
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
That's what I thought you meant. I don't know if I can iron out some of the tough spots in a way that suits everybody, or even anybody, but I'll try. Thanks for contributing your thoughts on what makes it difficult for you, I'll try to address some of those concerns. I used to work for a couple of truck drivers. One was in love with Kenworth tractors; the other thought Peterbilt ruled. They used to get into horrible battles over which was the best truck. When they realised neither of them was winning they'd fight over which engine was best. The one thought Cummins was top dog; the other thought if it wasn't Waukesha (now Detroit Diesel) it was junk. I couldn't understand how they could get so worked up and start cussing and throwing stuff at each other but eventually I realized that they spend years in their truck and eventually it becomes very close and personal to them. When someone else criticizes what they call home for a good part of their life it offends them. Ones choice of OS can become a very close and personal thing too if one spends enough time with it. BOINC <<--- credit whores, pedants, alien hunters |
|
Send message Joined: 2 Jan 09 Posts: 303 Credit: 7,322,550,090 RAC: 16,779 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
That's why I suggested they adopt a strategy of detecting whether the system has a touch screen or not and then give either the Metro look or the 'old' look, but no one liked that idea at the time, seems they are now revisiting the whole default 'Metro' look in the next update. I can see how going to a homogeneous OS for every type of system would work, but they must adapt it along the way to make it easy to use too. Essentially do the same thing in different ways depending on your device. |
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Here is a progress report on what I've discovered so far to let you know what I think I can do and how I think should proceed. Crunchuntu I mentioned the possibility that I might be able to derive a new OS I refer to as Crunchuntu from Ubuntu. I haven't abandoned Crunchuntu but after looking into in more detail it's obvious to me that it will take a long time. Plan B I also mentioned the possibility of a "do it all script" that you could download and run after you install Ubuntu. After exploring that idea and thinking about it I think the best way to do it would be 2 separate scripts. In memory of Dr. Seuss I refer to the 2 scripts as "Thing 1" and "Thing 2", now here's what they do. Thing 1 *runs on Windows 7/8, possibly XP too *written in Python (would require you to install Python which is relatively easy) or perhaps Windows Powershell (already exists on every Win 7/8 installation, no installation required) *you download the script and run it *the script automatically (no intervention from you) downloads the Ubuntu 12.04 ISO file, verifies the download is not corrupt via SHA256, burns the ISO to DVD, creates a new, blank partition for Ubuntu to install on later on whichever HDD you choose. *at that point you have a verified, bootable Ubuntu 12.04 DVD plus a partition to install it on else you have a log file explaining why the process failed
*runs on Ubuntu, in other words you install Ubuntu 12.04 using the DVD Thing1 created for you then you download and run Thing 2 *automatically downloads BOINC and installs it for you on your account so that you have all the permissions required to configure BOINC easily with a cc_config.xml, app_info or whatever you need *configures BOINC to start automatically at boot (if you select that option) and creates clickable icons for starting and stopping the client and manager whenever you wish *configures itself to autostart at boot time (if you wish) so it can monitor all sorts of stuff and do many of the things BOINC fails to do for example kill tasks that run for 11 days when they should require no more than 11 hours. *written in Python and has a GUI *could easily be included as is in Cruchuntu if Crunchuntu ever becomes a reality
BOINC <<--- credit whores, pedants, alien hunters |
|
Send message Joined: 11 Jan 13 Posts: 216 Credit: 846,538,252 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Sounds absolutely amazing. I've been wanting to teach myself Linux for a while now. My last attempt was with Debian and ended in failure with me trying desperately to give myself permission to install BOINC without any success. |
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hi Matt, Don't feel bad. If you don't know anything about Linux then trying to install it and BOINC and get it all running and crunching is more than most newbies can chew without choking. I did it in 2 steps. The first step was to firmly resolve that I wasn't going to just try it and see if I could make it work, I was going to learn it and make it work no matter what and no matter how long it took. Like Yoda said, "There is no try there is only do." Second step was to install Linux in a dual-boot configuration so I could switch back and forth. While I was at work or sleeping my computers ran Windows so they could crunch. After work I booted Linux and spent an hour a day learning how to use it from a book I bought and from online tutorials and forums. A couple months later I knew enough about Linux to install BOINC on it. It took another week to learn how to start BOINC. A month later I deleted the Windows partition on my 2 machines and booted only Linux. That wasn't the end of the learning process and I knew it but I also knew I would never have to revert to Windows again. Finally, I was free. Hopefully I can help get newbies up and crunching quickly by automating a few things for them. After they're crunching, if they want, they can take time to learn Linux so they can do more (eg. web surf, email, etc.). Or they can just let it crunch and do all their regular stuff on a Windows machine. My objective is not really to convert anybody to Linux. Only you can convert yourself. All I want is to make it possible for Vista/7/8 users to be able to crunch GPUgrid on Linux so they aren't hampered by WDDM. If that's all they ever do with Linux then I will have succeeded. If they become a full convert then that will be a success they can call their own, not mine. BOINC <<--- credit whores, pedants, alien hunters |
Carlesa25Send message Joined: 13 Nov 10 Posts: 328 Credit: 72,619,453 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hi Dagorath: The idea is very good and actually facilitate people as installing BOINC on Linux to automate the steps in a tutorial detailing only. One suggestion, to advise colleagues own experience, the problem is also have many users with Nvidia drivers and Coolbits, initially with the installation and configuration, then after the update. If not resolved and clear guidelines are given to this issue, it is useless to have Boinc installed correctly. |
|
Send message Joined: 2 Jan 09 Posts: 303 Credit: 7,322,550,090 RAC: 16,779 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Here is a progress report on what I've discovered so far to let you know what I think I can do and how I think should proceed. If time is of the essence I would do 'thing 2' and just include a small text file of where to find the version of Ubuntu you are basing your script on and how to download and burn it to a cd/dvd. MOST windows folks can do that much on their own, if they know where to look for the downloads. The after Linux is installed, with some pointers from your text file as to disk sizing etc, then your script takes over and BOOM everything is setup and running and then can begin crunching. One small suggestion would be to include a 2nd script for those that use a non crunching capable gpu to start with then add one later on, to automatically set it up with the latest drivers etc to get it up and crunching. Have you seen this yet: http://www.gpugrid.net/forum_thread.php?id=2871 It is a how to from the FAQ's to install Ubuntu Linux here at gpugrid. You might get with them to add your final choice(s) in that FAQ. Maybe an additional thread in there, or even a set of lines at the bottom, so users can do this if you want to do it yourself or download that set of scripts to do it all automatically. Wow that reads badly but I hope you get what I am trying to say! Giving users a choice to just download Crunchuntu is still of course the best ultimate choice for most I think, as most just want to click and crunch. But baby steps is not a bad idea for you to see how hard this is ultimately going to be to maintain over the long term. |
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hi Carlesa, You are correct, as usual :-) The drivers and Coolbits are difficult for newbies and I know because it took me a long time to understand it. I plan to automate drivers and Coolbits via a script but I forgot to mention it in my last post. Automating driver installation and driver upgrades and downgrades on stock Ubuntu 12.04 has been one of the more difficult parts of my task due to Unity but yesterday I finally found the advice I need to make it work. I haven't coded it yet but I'm confident it will work nicely and save a lot of work and confusion for everyone. Thanks for your interest in my project and I hope you continue to share your excellent advice and experience. BOINC <<--- credit whores, pedants, alien hunters |
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hi Mikey, Good point but just a few minutes after I outlined Thing 1, I recalled a problem mentioned by captainjack a few weeks ago. The problem is that if Linux is installed alongside Win7 in a dual-boot configuration in the usual way then Win7 won't install updates. The good news is that it's very easy to install it the way Win7 wants it. Captainjack's post provides a link to an excellent online tutorial, complete with step-by-step screenshots of every detail, that shows you exactly how to: 1) shrink an existing partition to make room for the Linux partitions (assuming you don't have unallocated space on your HDD) 2) install Ubuntu 12.04 3) install and run an app on Win7 that tweaks the boot partitions in a way that convinces Win7 it's OK to install updates The upshot of all that is Thing 1 no longer needs to do the repartitioning step so that saves me a lot of time coding Thing 1. All it has to do is D/L the Ubuntu 12.04 ISO, verify the download, burn the DVD then verify the burn. I think I can do that with an easy to write DOS batch file, no need for the user to install Python and no need for me to learn Windows Powershell. You are correct when you say those steps are easy to do manually but I've seen many newbies screw it up and end up with the wrong version or a bugged version because they didn't verify the integrity of the download or else didn't verify the burn. That creates problems for anybody trying to help them figure out what's going wrong. Nobody needs that. A proper DVD is the foundation upon which all that follows rests and it's best for all concerned to make absolutely sure that foundation is solid. So now I am in the process of setting up a rig with a fresh Win7 install on it so I can verify that Thing 1 works after I code it and that the online tutorial pointed to by captainjack works. When it works for me then I want a few other people to test it and verify that it works for them too. Are you down for a little testing, Mikey? I know you don't need Thing 1 but that makes you the perfect tester... if Thing 1 doesn't work for you you'll have a much better idea of where it's failing than someone who has never burned an ISO to DVD before. Edit added: BTW, here is the post by captainjack I mentioned. The URL for the tutorial he suggested is http://www.linuxbsdos.com/2012/05/17/how-to-dual-boot-ubuntu-12-04-and-windows-7/. @captainjack: if you're listening in, thanks again for the advice in that post and the link to the tutorial BOINC <<--- credit whores, pedants, alien hunters |
|
Send message Joined: 11 Jul 09 Posts: 1639 Credit: 10,159,968,649 RAC: 2 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
If it helps, I'm reasonably experienced in Windows (I can burn an ISO :P), but I am an utter and complete n00b in Linux. I have a Windows 7 Pro box (Dell OEM install) which I deliberately bought as a sacrificial OS to test some problems with the BOINC Windows installer and account permissions, when Rom Walton eventually has time to devote to that again - it has a factory recovery image of the OS in (I'd better check this) a separate partition of the hard disk. It's an i5 'Haswell', and the integrated Intel HD 4600 is crunching happily on projects that support it. A twitching in my wallet suggests that it might acquire a GTX 750 Maxwell of some description next month, so that would make it a candidate for Thing 2 as well. I also made a contribution to Adaptiva's "Parallella" project, so I should be receiving a release pack in due course. All things considered, this might be the year I start to learn Linux, so going along for the ride with Thing 1 and Thing 2 could be a way to do it. |
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Richard, Earning your interest tells me I am at least headed in the right general direction if not on precisely the right track. I would be grateful to have your considerable talent providing some feedback or whatever you have time for. There are a lot of signs telling me this is a good time to learn some Linux. With the advent of VT-d, maturation of virtual machine software, the tendency for project devs and researchers to prefer Linux and success (mostly) of Rom's VBox wrapper at T4T, I have a feeling more projects will be providing only Linux applications in the future. They will run on virtual Linux machines on everything from Macs to Unix machines, PCs to huge mainframes and relieve project devs of the chore of developing and maintaining apps for 3 or more OS's. Yes, a god time to learn Linux if I read the signs right. Slight change of topic: I have a new i7-4771 Haswell arriving this week, with VT-d, and I'm interested in experimenting with virtualizing GPUs. BOINC <<--- credit whores, pedants, alien hunters |
|
Send message Joined: 11 Jul 09 Posts: 1639 Credit: 10,159,968,649 RAC: 2 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Slight change of topic: Did you see Crunching in a Virtual Machine at Einstein? |
MJHSend message Joined: 12 Nov 07 Posts: 696 Credit: 27,266,655 RAC: 0 Level ![]() Scientific publications ![]()
|
I understand that GPU virtualisation is only supported for Tesla K20 and Grid GPUs. Matt |
|
Send message Joined: 26 Jun 09 Posts: 815 Credit: 1,470,385,294 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Dagorath, I go along with your project to follow its progress and make good use (parts) of it. I have Linux running on a system to test it. The most frustrating thing to me is that it is my PC, I have installed the OS and it are my files. But installing things wrongly then I am not the owner and do not have the right to go the the directories I made myself on my own PC! Please keep that in mind when making scripts. Thanks. Greetings from TJ |
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
From the discussion in the thread Richard linked to (thanks Richard, interesting read) at Einstein it sounds like there are 2 different techniques and I hope I have the terminology right: GPU virtualization and GPU passthrough. In the discussion at Einstein they were able to get GPU passthrough working on Xen with CPUs that have VT-d. They got it to work mid to high end AMD cards and OpenCL. They said NVIDIA doesn't support passthrough in their drivers on consumer grade cards but do support it (or intend to support it?) on Tesla K2 and Grid GPUs At least that is my take on the conversation but I had time for only 1 quick read. The other thing they mentioned regarding NVIDIA and passthrough is that consumer grade cards such as we use here at GPUgrid can be hacked to identify themselves as Teslas and Quadros to the NVIDIA drivers. I've read the procedure for the hack and it's not easy but I think I could do it. I was thinking of doing it to unleash DP power for Asteroids@home but now I have a second reason. BOINC <<--- credit whores, pedants, alien hunters |
MJHSend message Joined: 12 Nov 07 Posts: 696 Credit: 27,266,655 RAC: 0 Level ![]() Scientific publications ![]()
|
Howdo - sorry to hijack the thread. I've made you all a live Linux USB disk. The thread about it is here: http://www.gpugrid.net/forum_thread.php?id=3642 Matt |
|
Send message Joined: 16 Mar 11 Posts: 509 Credit: 179,005,236 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Dagorath, I go along with your project to follow its progress and make good use (parts) of it. I have Linux running on a system to test it. It sounds like you installed BOINC from the distro repository instead of installing it using the Berkeley installer. That's what happens when you install from the distro repository. It creates a user named boinc and installs BOINC on the account owned by the user named boinc. By default nobody but the superuser (aka root) and boinc can modify files that belong to boinc. Your regular account does not have permission to modify has boinc's files or install anything (a cc_config.xml) in any of boinc's directories (folders). If you could login as boinc you could modify boinc's files but boinc doesn't have a password so you cannot login as boinc. One solution is for you to become root. root is the name of the system administrator or superuser. When you are root you have permission to modify the files of any user who has an account on the computer. As root you can install a cc_config.xml, or an app_info.xml or whatever you want to do. root is all powerful. Unfortunately there a few things you need to learn in order to become root and add files to boinc's folders. If you Google "Linux file permissions" or similar topics you will find excellent online articles on how to work with Linux file permissions and user privileges. There is a better solution that will not require you to become root or deal with permissions. Instead of using the BOINC installer from distro repository, Thing 2 will download and use the Berkeley installer which installs BOINC on your account instead of on the account of a user named boinc. That way you, not a user named boinc, will own all of the files and folders BOINC client and BOINC manager use. You won't have to become root to add a cc_config.xml or modify client_state.xml. You will have complete control over BOINC's files. They will all be located in one sub-directory of your home directory where they will be easy for you to locate and work with. Will that work for you, TJ? BOINC <<--- credit whores, pedants, alien hunters |
skgivenSend message Joined: 23 Apr 09 Posts: 3968 Credit: 1,995,359,260 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
I think 'full' access is limited to, Quadro Series: Quadro 6000, Quadro 5000, Quadro 4000 GRID Series: GRID K2, GRID K1 M-Class: M2070-Q
FAQ's HOW TO: - Opt out of Beta Tests - Ask for Help |
|
Send message Joined: 26 Jun 09 Posts: 815 Credit: 1,470,385,294 RAC: 0 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Yes that works for me Dagorath. Having Linux installed and play with it helps to understand the OS. I am not crunching with it yet, the rig has not a GPU capable for it. Use it to become familiar with it. Greetings from TJ |
|
Send message Joined: 2 Jan 09 Posts: 303 Credit: 7,322,550,090 RAC: 16,779 Level ![]() Scientific publications ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]()
|
Hi Mikey, Sure, I have a box I am willing to sacrifice for testing. My main testing box is only a dual core so I hope that is okay, but I could do a quad if I need to. If you need one with a gpu it will have to be the dual core for now. Are you think Nvidia gpu's only, or are you thinking Boinc wide and ATI and Nvidia gpu's will both work. Or is all that too early yet? How big of a hard drive will I need in the machine, right now I have a 160gb one with over 90gb free in my test box, but I have a small stack of other drives I could add in if needed. |
©2026 Universitat Pompeu Fabra