git clone not working.

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

git clone not working.

Peter Helfer
Hi!

I just wanted to know what is going on with the kernel repo - whenever I run git on my Mac 10.5, it turns up following problem:

[foobar:/Users/Shared/Kernel]> git clone git://git.openmoko.org/git/kernel.git linux-2.6 
Initialized empty Git repository in /Users/Shared/Kernel/linux-2.6/.git/
remote: Generating pack...
remote: Done counting 893800 objects.
remote: fatal: remote: packfile ./objects/pack/pack-ae90d81e3c7e27be22a87e87e4e86f78bf2fcfcc.pack cannot be mapped.remote: 
remote: aborting due to possible repository corruption on the remote side.
fatal: early EOF
fatal: index-pack died with error code 128
fetch-pack from 'git://git.openmoko.org/git/kernel.git' failed.


is there a daily source repo (no, not the built ones!) where I could d/l the patched kernel sources with all the changes ?

Regards, Peter

Reply | Threaded
Open this post in threaded view
|

Re: git clone not working.

Sean McNeil
Hi Peter,

Do you have a directory called linux-2.6 already? Does it have the linux
git in there?  Please use a non-existent directory name like:

git clone git://... linux-openmoko

Peter Helfer wrote:

> Hi!
>
> I just wanted to know what is going on with the kernel repo - whenever
> I run git on my Mac 10.5, it turns up following problem:
>
> [foobar:/Users/Shared/Kernel]> git clone
> git://git.openmoko.org/git/kernel.git
> <http://git.openmoko.org/git/kernel.git> linux-2.6
> Initialized empty Git repository in /Users/Shared/Kernel/linux-2.6/.git/
> remote: Generating pack...
> remote: Done counting 893800 objects.
> remote: fatal: remote: packfile
> ./objects/pack/pack-ae90d81e3c7e27be22a87e87e4e86f78bf2fcfcc.pack
> cannot be mapped.remote:
> remote: aborting due to possible repository corruption on the remote side.
> fatal: early EOF
> fatal: index-pack died with error code 128
> fetch-pack from 'git://git.openmoko.org/git/kernel.git
> <http://git.openmoko.org/git/kernel.git>' failed.
>
>
> is there a daily source repo (no, not the built ones!) where I could
> d/l the patched kernel sources with all the changes ?
>
> Regards, Peter
>

Reply | Threaded
Open this post in threaded view
|

Re: git clone not working.

Cesar Eduardo Barros
In reply to this post by Peter Helfer
Peter Helfer escreveu:

> I just wanted to know what is going on with the kernel repo - whenever I
> run git on my Mac 10.5, it turns up following problem:
>
> [foobar:/Users/Shared/Kernel]> git clone
> git://git.openmoko.org/git/kernel.git
> <http://git.openmoko.org/git/kernel.git> linux-2.6
> Initialized empty Git repository in /Users/Shared/Kernel/linux-2.6/.git/
> remote: Generating pack...
> remote: Done counting 893800 objects.
> remote: fatal: remote: packfile
> ./objects/pack/pack-ae90d81e3c7e27be22a87e87e4e86f78bf2fcfcc.pack cannot
> be mapped.remote:
> remote: aborting due to possible repository corruption on the remote side.
> fatal: early EOF
> fatal: index-pack died with error code 128
> fetch-pack from 'git://git.openmoko.org/git/kernel.git
> <http://git.openmoko.org/git/kernel.git>' failed.
>
>
> is there a daily source repo (no, not the built ones!) where I could d/l
> the patched kernel sources with all the changes ?

There is a full mirror of that git repository at
http://repo.or.cz/w/linux-2.6/openmoko-kernel.git (clone from
git://repo.or.cz/linux-2.6/openmoko-kernel.git or
http://repo.or.cz/r/linux-2.6/openmoko-kernel.git).

--
Cesar Eduardo Barros
[hidden email]
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: git clone not working.

andy green-3
In reply to this post by Peter Helfer
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Somebody in the thread at some point said:
| Hi!
|
| I just wanted to know what is going on with the kernel repo - whenever I
| run git on my Mac 10.5, it turns up following problem:
|
| [foobar:/Users/Shared/Kernel]> git clone
| git://git.openmoko.org/git/kernel.git
| <http://git.openmoko.org/git/kernel.git> linux-2.6
| Initialized empty Git repository in /Users/Shared/Kernel/linux-2.6/.git/
| remote: Generating pack...
| remote: Done counting 893800 objects.
| remote: fatal: remote: packfile
| ./objects/pack/pack-ae90d81e3c7e27be22a87e87e4e86f78bf2fcfcc.pack cannot
| be mapped.remote:
| remote: aborting due to possible repository corruption on the remote side.
| fatal: early EOF
| fatal: index-pack died with error code 128
| fetch-pack from 'git://git.openmoko.org/git/kernel.git
| <http://git.openmoko.org/git/kernel.git>' failed.
|
|
| is there a daily source repo (no, not the built ones!) where I could d/l
| the patched kernel sources with all the changes ?

Yes it's borked somehow, I see the same.

I nuked the kernel.git dir on there are push my local one, that seems to
have fixed it.  I will upload a few more branches now.

- -Andy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAki/KxsACgkQOjLpvpq7dMqUkQCfRybqVLDD6sKh8Fls2FjI5cXG
NlgAoJXkTqxPw5p1SJK/0hVAH7/PbKBg
=QDWt
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: git clone not working.

Cesar Eduardo Barros
Andy Green escreveu:
> Yes it's borked somehow, I see the same.
>
> I nuked the kernel.git dir on there are push my local one, that seems to
> have fixed it.  I will upload a few more branches now.

cesarb@delta:~/scratch/src/git/openmoko/kernel.git$
PATH=~/opt/git/bin:$PATH git mirror
remote: Generating pack...
remote: Done counting 274 objects.
remote: Result has 221 objects.
remote: Deltifying 221 objects.
remote:  100% (221/221) done
remote: Total 221, written 221 (delta 184), reused 0 (delta 0)
Receiving objects: 100% (221/221), 54.79 KiB | 33 KiB/s, done.
Resolving deltas: 100% (184/184), completed with 52 local objects.
 From git://git.openmoko.org/git/kernel
  + dd32338...e20ce2b andy       -> andy  (forced update)
  + 2ceaa01...0efe573 debug      -> debug  (forced update)
Removing refs/heads/andy-tracking
Removing refs/heads/mokopatches
Removing refs/heads/debug-tracking
Removing refs/heads/zecke
Removing refs/heads/stable-2.6.26
Removing refs/heads/mokopatches-tracking
Removing refs/heads/stable-tracking
Removing refs/heads/zecke-2.6.26
Removing refs/heads/mokopatches-2.6.26
Removing refs/heads/willie
Removing refs/heads/2.6.26

Looks like we lost almost all the branches. We only have these ones
left: andy debug master stable

Unfortunately, the reflog seems to be deleted together with the
corresponding ref; I'll probably make later a patch to git to avoid
that. I have a copy of all of them (as of a few days ago, but IIRC only
stable and andy has changed in the past few days) in the "origin" remote
of my cpufreq repository:

cesarb@delta:~/work/openmoko/cpufreq/linux-2.6-s3c2410-cpufreq$
PATH=~/opt/git/bin:$PATH git ls-remote . remotes/origin/*
bce7f793daec3e65ec5c5705d2457b81fe7b5725        refs/remotes/origin/2.6.26
796aadeb1b2db9b5d463946766c5bbfd7717158c        refs/remotes/origin/HEAD
dd323382faf5c169960659d5d78337f06b3ea222        refs/remotes/origin/andy
e72b7a04a72c699a93040065c6353376da3b4f24
refs/remotes/origin/andy-tracking
2ceaa014cb53a4cd4c75d889bf2feee214c1426b        refs/remotes/origin/debug
43999fa462d86a468d4778c0e2a5ea155d452419
refs/remotes/origin/debug-tracking
796aadeb1b2db9b5d463946766c5bbfd7717158c        refs/remotes/origin/master
ce8eaeee7875e668be4c2888fa05a61a5af0dd98
refs/remotes/origin/mokopatches
38801452b334d3591eddadbfa4a31529b8957513
refs/remotes/origin/mokopatches-2.6.26
a819ba15d0390b4bb3671e49b4ec9e3703681c6a
refs/remotes/origin/mokopatches-tracking
d744c88c149269b95ec068c8615e492375415d6d        refs/remotes/origin/stable
557b4763b753b1620e922864555a4abf851372ba
refs/remotes/origin/stable-2.6.26
303ff417ef62753985b231031b69a48c34911669
refs/remotes/origin/stable-tracking
e87e44f23b889b529107d508e5d52a5b88f1eb51        refs/remotes/origin/willie
fa269b44512a03523b164c3cebc20312748c524b        refs/remotes/origin/zecke
c634ff11e8c5e087d784219698bea29abbc1697a
refs/remotes/origin/zecke-2.6.26

--
Cesar Eduardo Barros
[hidden email]
[hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: git clone not working.

andy green-3
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Somebody in the thread at some point said:

| Looks like we lost almost all the branches. We only have these ones
| left: andy debug master stable

It's OK I have them all and will push them back up, I just stuck the
critical ones back until now.

| Unfortunately, the reflog seems to be deleted together with the
| corresponding ref; I'll probably make later a patch to git to avoid
| that. I have a copy of all of them (as of a few days ago, but IIRC only
| stable and andy has changed in the past few days) in the "origin" remote
| of my cpufreq repository:
|
| cesarb@delta:~/work/openmoko/cpufreq/linux-2.6-s3c2410-cpufreq$
| PATH=~/opt/git/bin:$PATH git ls-remote . remotes/origin/*
| bce7f793daec3e65ec5c5705d2457b81fe7b5725        refs/remotes/origin/2.6.26
| 796aadeb1b2db9b5d463946766c5bbfd7717158c        refs/remotes/origin/HEAD
| dd323382faf5c169960659d5d78337f06b3ea222        refs/remotes/origin/andy
| e72b7a04a72c699a93040065c6353376da3b4f24 refs/remotes/origin/andy-tracking
| 2ceaa014cb53a4cd4c75d889bf2feee214c1426b        refs/remotes/origin/debug
| 43999fa462d86a468d4778c0e2a5ea155d452419
refs/remotes/origin/debug-tracking
| 796aadeb1b2db9b5d463946766c5bbfd7717158c        refs/remotes/origin/master
| ce8eaeee7875e668be4c2888fa05a61a5af0dd98 refs/remotes/origin/mokopatches
| 38801452b334d3591eddadbfa4a31529b8957513
| refs/remotes/origin/mokopatches-2.6.26
| a819ba15d0390b4bb3671e49b4ec9e3703681c6a
| refs/remotes/origin/mokopatches-tracking
| d744c88c149269b95ec068c8615e492375415d6d        refs/remotes/origin/stable
| 557b4763b753b1620e922864555a4abf851372ba refs/remotes/origin/stable-2.6.26
| 303ff417ef62753985b231031b69a48c34911669
| refs/remotes/origin/stable-tracking
| e87e44f23b889b529107d508e5d52a5b88f1eb51        refs/remotes/origin/willie
| fa269b44512a03523b164c3cebc20312748c524b        refs/remotes/origin/zecke
| c634ff11e8c5e087d784219698bea29abbc1697a refs/remotes/origin/zecke-2.6.26
|

I just moved that dir out of the way rather than really deleted it so I
have all these guys still.

What do you recommend, just cp them back to these dirs from the busted
repo image?

- -Andy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iEYEARECAAYFAki/QecACgkQOjLpvpq7dMqaRwCeM8fBmVSZoylijL0RHL/zIglD
w6UAnA3WAzZ6pgATwLTWRW7NpWqJvUrb
=p48Q
-----END PGP SIGNATURE-----

Reply | Threaded
Open this post in threaded view
|

Re: git clone not working.

Cesar Eduardo Barros
Andy Green escreveu:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Somebody in the thread at some point said:
>
> | Looks like we lost almost all the branches. We only have these ones
> | left: andy debug master stable
>
> It's OK I have them all and will push them back up, I just stuck the
> critical ones back until now.
>
> | Unfortunately, the reflog seems to be deleted together with the
> | corresponding ref; I'll probably make later a patch to git to avoid
> | that. I have a copy of all of them (as of a few days ago, but IIRC only
> | stable and andy has changed in the past few days) in the "origin" remote
> | of my cpufreq repository:
>
> I just moved that dir out of the way rather than really deleted it so I
> have all these guys still.
>
> What do you recommend, just cp them back to these dirs from the busted
> repo image?

They could either be on files at refs/heads/ or on the packed-refs file,
making it a bit harder to cp (not much; you just have to copy both, and
push your new refs again[1]). However, the missing objects (from the
broken pack ae90d81e3c7e27be22a87e87e4e86f78bf2fcfcc and possibly other
packs -- we don't know what else could be corrupted) are a bit harder to
recover.

If you or someone else has a clone with the correct references (check
them against git ls-remote --heads . on the broken repository), the
easiest way is to push them all, and nothing else would need to be done.

The second easiest way would to recover would be to create a copy of the
damaged repository, copy a set of working packs and loose objects from
someone who has them (you could try downloading everything from
http://repo.or.cz/r/linux-2.6/openmoko-kernel.git/objects/, which is
updated hourly and doesn't seem to have been repacked yet[2]), remove
the broken packs and/or objects (we already know of at least one)[3],
and check the result with git fsck --full. If it works, copy the
resulting set of objects and references to the new repository, copy the
refs as above, and fsck the new repository (just to be sure); or as an
alternative simply put the broken (now fixed) repository back in place.

The last step would be to run git update-server-info (since you skipped
any hook which would run it).

After it's fixed, do a git fsck --full on _all_ the other repositories
in that machine. git repositories shouldn't get corrupted that easily,
and it almost always points to hardware problems. You could also either
update git on the server to at least 1.6.0 or set pack.indexVersion to 2
(the new default on 1.6.0) and do a full repack of everything; the
resulting packs have extra protection against corruption, but are
incompatible with git 1.4.4.4[4] or earlier when fetching over http
(fetching over the native git protocol works, and git 1.4.4.5 also
works; the error message is a very obscure "non-monotonic index" one).


[1] Or you can use git ls-remote --heads . on the broken repository and
use git update-ref on the new repository to recreate each one by hand,
which also updates the reflog.

[2] My own copy on my machine should be a little behind but hasn't been
repacked for sure.

[3] Very recent versions of git should be able to skip the damaged
objects within the pack and recover the rest while doing a full repack,
so on them a full repack might be an alternative to removing the damaged
packs and objects, but if it works it's simpler to remove them.

[4] Only Debian etch (the current stable) still has that old version.
git 1.4.4.5 was made especially for them; the only change is the ability
to read (but not write) the version 2 pack index.

--
Cesar Eduardo Barros
[hidden email]
[hidden email]

roh
Reply | Threaded
Open this post in threaded view
|

Re: git clone not working. / an admin's rant

roh
Administrator
Cesar Eduardo Barros wrote:

> After it's fixed, do a git fsck --full on _all_ the other repositories
> in that machine. git repositories shouldn't get corrupted that easily,
> and it almost always points to hardware problems.

nope.. it also is rotten enough software to run totally amok when not
getting a socket or some other unix resource due to to hitting a
resource limit (which didn't happen in this case at hand, but i've seen
that before)
no broken hw necessary.

> You could also either
> update git on the server to at least 1.6.0 or set pack.indexVersion to 2
> (the new default on 1.6.0) and do a full repack of everything; the
> resulting packs have extra protection against corruption, but are
> incompatible with git 1.4.4.4[4] or earlier when fetching over http
> (fetching over the native git protocol works, and git 1.4.4.5 also
> works; the error message is a very obscure "non-monotonic index" one).

GAAAH.. sorry, but WTF? do they break compatibility on purpose all the time?
just for the record.. 1.6.0 was released on 17. of august this year. so
its not in _ANY_ stable distro yet. (and 1.6.0.1 on 24th..)

git 1.4.4.5 doesnt exist it seems.
atleast it doesnt show on
http://www.kernel.org/pub/software/scm/git/
at all

i would rule out hw problems, since git runs in a openvz-vm of its own,
and none of the other vms or the host showed recent trouble of that kind.
i rather guess that its (again) git which 'sucks around big time'

sometimes one thinks these guys never had to deal with the real world
outside their heads where people even need to tunnel through https to
get broken proxies not breaking their http and git:// does not work at
all. (and corporate policy hindering em to fix it).

one wouldn't believe how often one hears of such problems once working
as admin ;)

means: currently we run git version 1.4.4.4 on that machine.

if we want to change that, it means somebody again needs to 'bless the
version used' and take care of _maintaining_ that.

currently its us admins maintaining that VM, keeping it up to date with
security patches, while keeping as much as possible on stock versions.
the timestamps on the last few git relases do not make any serious
impression of 'a version we can run some time, without constant taking
care manually'

from time to time i did a local repack like this:
https://admin-trac.openmoko.org/trac/ticket/1492
since there is a warning about some breakage, thats not a cron. (yet)

i would stop doing this, but repo grows substantially and updates as
well as clones are _extremely_ slow when not doing a repack from time to
time.

the even more strange thing is that i didnt run a repack at all in
september yet. afaik.

so: sorry.. aslong as it breaks compatibility with debian stable you
guys need a better idea than 'update to head, we are sure its broken
differently!!11!' *sigh*

too sad.. i started liking git. but when they continue to nag people
with repeated incompatibilities/breakage on the wire or on fs, its are
not worth anything more than mtn, which drove enough people mad beyond
repair with its 'not performance'

> [3] Very recent versions of git should be able to skip the damaged
> objects within the pack and recover the rest while doing a full repack,
> so on them a full repack might be an alternative to removing the damaged
> packs and objects, but if it works it's simpler to remove them.

we need repacks from time to time anyways.

> [4] Only Debian etch (the current stable) still has that old version.
> git 1.4.4.5 was made especially for them; the only change is the ability
> to read (but not write) the version 2 pack index.

my conclusion is that i 'do not change anything', without somebody
having a _very_ convincing explanation why, and what.

upping to 1.6.0 means breaking the wire protocol and their
update-frequency tells me: 'i will not be the idiot running behind these
guys all the time'.

so git-guys, please fix that. find a working version which
 a) doesnt break the repo
 b) can be repacked automated without breaking the repo
 c) does not need manual rebuilding of git packages every half month.
 d) doesnt break compatibility with 1.4 and 1.5 git series. (i guess one
could drop 1.4 compat end of the year or a bit later, but not now. 1.5
probably compat will need to stay another year or 2)


sorry about being a bit harsh, but watching all this madness while 'svn
on fsfs just works' (in most (smaller) usecases ;), makes one quite sad.
(especially because it eats peoples time with stuff which should be
'base technology' and not something changing all the time.

--

Joachim Steiger
Openmoko Central Services