790765 - get mozbase tests running on m-c

Status: RESOLVED FIXED

(Testing :: Mozbase, defect)

Description

[Jeff Hammel]

We currently mirror all of mozbase to m-c, including the tests.
However, we don't actually run the tests.  We should!  Probably as
part of the build tests.

Comment 1

[Jeff Hammel]

Should probably be added here: http://mxr.mozilla.org/mozilla-central/source/build/Makefile.in#105 .

Comment 2

[(not currently active) Ted Mielczarek]

You can just add a PYTHON_UNIT_TESTS assignment to any Makefile.

Comment 3

[Jeff Hammel]

Attachment: WIP

So the good news is: I got test.py running locally. (Ignore badtest.py; it is only for testing purposes and will be deleted before checkin to m-c.) The bad news is: the mozprofile tests are broken as they demand the `mozprofile` executable (bug 837897). This isn't available even on the non-windows platforms as we don't run setup.py install/develop.

Comment 4

[Jeff Hammel]

So we're definitely going to need test.py updated as currently the exit code is not correct (see https://bugzilla.mozilla.org/show_bug.cgi?id=818967 ) as well as moztest, I think . I'll break these off from bug 838074 depending on what my runs show

Comment 5

[Jeff Hammel]

Also, we should add to our documentation on how to run these tests on m-c. I've just been using `make check` but that is horribly inefficient

Comment 6

[(not currently active) Ted Mielczarek]

With this patch you should be able to just run "make check" in $objdir/testing/mozbase. If we fixed bug 837824 you ought to be able to just "mach check testing/mozbase".

Comment 7

[Jeff Hammel]

Ah, thanks ted.  It appears I have forgotten how make works

Comment 8

[Jeff Hammel]

(In reply to Jeff Hammel [:jhammel] from comment #4)
> So we're definitely going to need test.py updated as currently the exit code
> is not correct (see https://bugzilla.mozilla.org/show_bug.cgi?id=818967 ) as
> well as moztest, I think . I'll break these off from bug 838074 depending on
> what my runs show

Confirmed that this fixes.  We now fail because we should! (locally)

Comment 9

[Jeff Hammel]

Attachment: 4 lines of code. 4 weeks. 1 developer.

Likely the final form of this patch.  Works locally with mozprofile patch to be uploaded subsequently.  Will put up for review once upstream is fixed and green on try.

Comment 10

[Jeff Hammel]

Attachment: fixes to mozprofile

Interim patch for mozprofile for testing.  Not to be pushed to m-c

Comment 11

[Jeff Hammel]

pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=4da37c007e0f

I expect this not to work ;) OTOH, I don't exactly know what will go wrong.  It is likely that the mozprocess tests will fail, those damn bastards

Comment 12

[Mozilla RelEng Bot]

Try run for 4da37c007e0f is complete.
Detailed breakdown of the results available here:
    https://tbpl.mozilla.org/?tree=Try&rev=4da37c007e0f
Results (out of 18 total builds):
    exception: 2
    success: 15
    warnings: 1
Builds (or logs if builds failed) available at:
http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-4da37c007e0f

Comment 13

[Jeff Hammel]

A pleasant surprise, for a change :)  Don't know what the windows purples are about but it looks like infra.  Since this will all have to be retried again if I'm wrong it should be caught.

Comment 14

[Jeff Hammel]

Attachment: unbitrot

to push to try

Comment 15

[Jeff Hammel]

(In reply to Jeff Hammel [:jhammel] from comment #14)
> Created attachment 729769 [details] [diff] [review]
> unbitrot
> 
> to push to try

https://tbpl.mozilla.org/?tree=Try&rev=3749a1174a55

Comment 16

[William Lachance (:wlach)]

Comment on attachment 729769 [details] [diff] [review]
unbitrot

LGTM, as long as it works :)

Comment 17

[Jeff Hammel]

Bug 855863 details the work to make devicemanager+tests work on windows. While initially hoped to be a quick fix, it looks like it is not that easy.

Should we disabled the dm tests on windows as an interim fix?  If not, we should figure out and make actionable what we're actually doing.

Comment 18

[Jonathan Griffin (:jgriffin)]

My vote would be to disable the dm tests on Windows.  We don't use dm on Windows, and I don't foresee any use cases for it, so trying to maintain Windows compatibility is probably not worth the effort.

Comment 19

[Jeff Hammel]

Tried this patch after the uplift of mozprofile, etc (bug 860091). Of course, the obvious idea that things would work....isn't :(

----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jhammel/mozilla/src/mozilla-central/testing/mozbase/mozprofile/tests/addonid.py", line 36, in test_addonID_xpi
    addon = a.addon_details(os.path.join(here, "addons", "empty.xpi"))
  File "/home/jhammel/mozilla/src/mozilla-central/testing/mozbase/mozprofile/mozprofile/addons.py", line 151, in addon_details
    doc = minidom.parse(os.path.join(addon_path, 'install.rdf'))
  File "/usr/lib/python2.7/xml/dom/minidom.py", line 1920, in parse
    return expatbuilder.parse(file)
  File "/usr/lib/python2.7/xml/dom/expatbuilder.py", line 922, in parse
    fp = open(file, 'rb')
IOError: [Errno 2] No such file or directory: '/home/jhammel/mozilla/src/mozilla-central/testing/mozbase/mozprofile/tests/addons/empty.xpi/install.rdf'

----------------------------------------------------------------------
Ran 102 tests in 124.637s

FAILED (errors=1)
make[1]: *** [run-test.py] Error 1
make[1]: Leaving directory `/home/jhammel/mozilla/src/obj-browser/testing/mozbase'
make: *** [check] Error 2

Well, its true...the file doesn't exist nor does it in mozbase canon master.  I've verified (ABICT) that the uplift was performed correctly.  Will poke a bit but owe a bunch of reviews

Comment 20

[Jeff Hammel]

oh, meant to note that this is not a problem in mozbase canon

Comment 21

[Jeff Hammel]

filed bug 863817 about better error checking

Comment 22

[Jeff Hammel]

Beh, for all that the file does not exist in m-c! :(  I'm not sure what happened...guess investigation is warranted :( :(

Comment 23

[Jeff Hammel]

Looking at it a bit without going into too much detail, I can't figure out why empty.xpi is missing.  It probably indicates an error in the mirroring script http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/generate_diff.py but I can't find evidence for that (or anything else) in short order.  Will file a follow up for the issue.

Adding empty.xpi does fix make check.  Will update the patch here and push to try.

Comment 24

[Jeff Hammel]

Attachment: make check + missing empty.xpi

Comment 25

[Jeff Hammel]

(In reply to Jeff Hammel [:jhammel] from comment #23)
> Looking at it a bit without going into too much detail, I can't figure out
> why empty.xpi is missing.  It probably indicates an error in the mirroring
> script
> http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/generate_diff.
> py but I can't find evidence for that (or anything else) in short order. 
> Will file a follow up for the issue.

bug 870867

> Adding empty.xpi does fix make check.  Will update the patch here and push
> to try.

https://tbpl.mozilla.org/?tree=Try&rev=750e00dbc353

Comment 26

[Andrew Halberstadt [:ahal]]

Comment on attachment 748039 [details] [diff] [review]
make check + missing empty.xpi

Review of attachment 748039 [details] [diff] [review]:
-----------------------------------------------------------------

Heh, splinter doesn't give any indication that binary files are added.. but it's there in the raw diff.

Comment 27

[Jeff Hammel]

try looks good except windows.  these look unrelated.  will push again

Comment 28

[Jeff Hammel]

ah, actual real errors:

====================================================================== ERROR: test_process_kill (test_mozprocess.ProcTest) Process is started, we kill it ---------------------------------------------------------------------- Traceback (most recent call last): File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\tests\test_mozprocess.py", line 187, in test_process_kill p.run() File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 621, in run self.proc = self.Process(self.cmd, **args) File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 76, in __init__ universal_newlines, startupinfo, creationflags) File "c:\mozilla-build\python27\Lib\subprocess.py", line 679, in __init__ errread, errwrite) File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\mozprocess\processhandler.py", line 194, in _execute_child cwd, startupinfo) File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\mozprocess\winprocess.py", line 184, in ErrCheckCreateProcess ErrCheckBool(result, func, args) File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\mozprocess\winprocess.py", line 51, in ErrCheckBool raise WinError() WindowsError: [Error 2] The system cannot find the file specified. ======================================================================

From https://tbpl.mozilla.org/php/getParsedLog.php?id=22844707&tree=Try&full=1#error0

Not really surprisingly, from the mozprocess tests.

And...

======================================================================
ERROR: test_existing_permissions_db_v2 (permissions.PermissionsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprofile\tests\permissions.py", line 172, in test_existing_permissions_db_v2
    self.verify_user_version(2)
  File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprofile\tests\permissions.py", line 154, in verify_user_version
    Permissions(self.profile_dir, self.locations_file.name)
  File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprofile\mozprofile\permissions.py", line 228, in __init__
    self._locations.read(locations)
  File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprofile\mozprofile\permissions.py", line 161, in read
    locationFile = codecs.open(filename, "r", "UTF-8")
  File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\obj-firefox\_virtualenv\lib\codecs.py", line 881, in open
    file = __builtin__.open(filename, mode, buffering)
IOError: [Errno 13] Permission denied: 'c:\\users\\cltbld\\appdata\\local\\temp\\tmpdcnjn9'

I have a feeling I've seen this before...

Comment 29

[Jonathan Griffin (:jgriffin)]

These are probably due to use of NamedTemporaryFile, which works differently on Windows than on posix systems.

Comment 30

[Gregory Szorc [:gps]]

We have mozfile.NamedTemporaryFile which allows you to open the file multiple times on Windows. It doesn't have all the semantics of NamedTemporaryFile, but it should work for normal use cases.

Comment 31

[Jeff Hammel]

As best I can tell, the mozprofile test failure may very well be NamedTemporaryFile: http://mxr.mozilla.org/mozilla-central/source/testing/mozbase/mozprofile/tests/permissions.py#29 .

The mozprocess test failure, OTOH, does not seem to be NamedTemporaryFile related.  My blind guess is that ... the executable is wrong? (somehow?)

I'll add some diagnostics to the mozprocess tests which will be good to have going forward anyway, try mozfile.NamedTemporaryFile for the mozprofile failure, and push to try again.

Comment 32

[Jeff Hammel]

Attachment: WIP

mostly for information gathering and to try a few fixes; perhaps not final form but i wanted to get a run off today.

https://tbpl.mozilla.org/?tree=Try&rev=de9a867fc684

Comment 33

[Jeff Hammel]

Don't get too excited;  I sadly didn't push the original patch along with it. boo :(

Comment 34

[Jeff Hammel]

one more time  https://tbpl.mozilla.org/?tree=Try&rev=3ace1d0f408e

Comment 35

[Jeff Hammel]

Aborted due to local failures:

======================================================================
ERROR: test_schema_version (bug785146.PermissionsTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jhammel/mozilla/src/mozilla-central/testing/mozbase/mozprofile/tests/bug785146.py", line 41, in test_schema_version
    perms.write_db(self.locations_file)
  File "/home/jhammel/mozilla/src/mozilla-central/testing/mozbase/mozprofile/mozprofile/permissions.py", line 247, in write_db
    for location in locations:
TypeError: 'NamedTemporaryFile' object is not iterable

----------------------------------------------------------------------
...speaking of missing methods....this needs an __iter__ (and a next)

======================================================================
ERROR: test_prefs_write (test_preferences.PreferencesTest)
test that the Preferences.write() method correctly serializes preferences
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jhammel/mozilla/src/mozilla-central/testing/mozbase/mozprofile/tests/test_preferences.py", line 245, in test_prefs_write
    os.remove(path)
TypeError: coercing to Unicode: need string or buffer, NoneType found

now that's just stupidity on my part...I think...  Anyway, some cleanup here

Comment 36

[Jeff Hammel]

ah, sadly not just my stupidity:

======================================================================
ERROR: test_prefs_write (test_preferences.PreferencesTest)
test that the Preferences.write() method correctly serializes preferences
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jhammel/mozilla/src/mozilla-central/testing/mozbase/mozprofile/tests/test_preferences.py", line 237, in test_prefs_write
    with mozfile.NamedTemporaryFile(suffix='.js', delete=False) as f:
TypeError: __init__() got an unexpected keyword argument 'delete'

Will ticket these.  Stictly speaking, the __iter__/next isn't necessary but the delete=False is needed.

Comment 37

[Jeff Hammel]

filed: bug 872138

Comment 38

[Jeff Hammel]

Just to explicitly state while some tests are running...

The patches made here to mozbase will be committed to mozbase core when they're in good/final form.

mozfile will be updated, bumped, and mirrored to m-c , potentially along with the above patches to mozbase

*THEN* the 4-line makefile patch can be committed, potentially alongside the missing .xpi file unless I happen to solve that mystery by then

Comment 39

[Jeff Hammel]

pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=e77535da2160

We all fail, but at least for the right reasons this time.  Maybe o_O . Output is interleaved (thank you windows) so it is difficult to tell what is really going on.  But to paste the relevant parts from https://tbpl.mozilla.org/php/getParsedLog.php?id=23006071&tree=Try&full=1:

"""
test_logging_enabled (requestlog.RequestLogTest) ... PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpp2t1r3\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpyjvxcj\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpatohwq\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmp8wxk_y\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmphnqnev\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpcs3ob1\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpcakb65\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpxdqkjd\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmp54q3ed\test.dmp this is some output PROCESS-CRASH | test.py | application crashed [Unknown top frame] Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpiqxxko\test.dmp this is some output ['make']: exit 2 stdout: make[2]: Entering directory `/e/builds/moz2_slave/try-w32-0000000000000000000000/build/testing/mozbase/mozprocess/tests' cl //Od //I "iniparser" //D "WIN32" //D "_WIN32" //D "_DEBUG" //D "_CONSOLE" //D "_UNICODE" //D "UNICODE" //Gm //EHsc //RTC1 //MDd //W3 //nologo //c //ZI //TC proclaunch.c proclaunch.c proclaunch.c(23) : warning C4996: '_swprintf': This function or variable may be unsafe. Consider using _swprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(539) : see declaration of '_swprintf' proclaunch.c(96) : warning C4996: 'strtok': This function or variable may be unsafe. Consider using strtok_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\string.h(166) : see declaration of 'strtok' proclaunch.c(106) : warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(366) : see declaration of 'sprintf' proclaunch.c(107) : warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(366) : see declaration of 'sprintf' proclaunch.c(120) : warning C4996: 'strtok': This function or variable may be unsafe. Consider using strtok_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details. C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\string.h(166) : see declaration of 'strtok' proclaunch.c(126) : warning C4101: 'cmd' : unreferenced local variable link //OUT:"proclaunch.exe" //INCREMENTAL //LIBPATH:"iniparser\\" //NOLOGO //DEBUG //SUBSYSTEM:CONSOLE //DYNAMICBASE //NXCOMPAT //MACHINE:X86 //ERRORREPORT:PROMPT iniparser.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib proclaunch.obj proclaunch.obj : fatal error LNK1112: module machine type 'x64' conflicts with target machine type 'X86' make[2]: Leaving directory `/e/builds/moz2_slavok ERROR
"""

This is almost assuredly not test_logging_enabled but with windows stderr/stdout interleaving combined with mozcrash not telling us what the application is, it is hard to say.

"""
test_remove_directory (test.TestRemoveTree) ... e/try-w32-0000000000000000000000/build/testing/mozbase/mozprocess/tests'

stderr:
cl : Command line warning D9038 : /ZI is not supported on this platform; enabling /Zi instead

cl : Command line warning D9007 : '/Gm' requires '/Zi'; option ignored

make[2]: *** [proclaunch] Error 88
make[2]: Target `default' not remade because of errors.


PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpp6n5cm\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmptjcpvr\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpenovey\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmp0z7wry\test.dmp
this is some output
PROCESS-CRASH | test.py | application crashed [Unknown top frame]
Crash dump filename: c:\users\cltbld\appdata\local\temp\tmpftohxr\test.dmp
this is some output
ok

======================================================================
ERROR: setUpClass (test_mozprocess.ProcTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\tests\test_mozprocess.py", line 95, in setUpClass
    cls.proclaunch = make_proclaunch(here)
  File "e:\builds\moz2_slave\try-w32-0000000000000000000000\build\testing\mozbase\mozprocess\tests\test_mozprocess.py", line 46, in make_proclaunch
    raise subprocess.CalledProcessError(process.returncode, command, stdout)
CalledProcessError: Command '['make']' returned non-zero exit status 2

----------------------------------------------------------------------
Ran 83 tests in 11.983s

FAILED (errors=1)
e:\builds\moz2_slave\try-w32-0000000000000000000000\build\config\rules.mk:188:0: command 'PYTHONDONTWRITEBYTECODE=1 e:/builds/moz2_slave/try-w32-0000000000000000000000/build/obj-firefox/_virtualenv/Scripts/python.exe e:/builds/moz2_slave/try-w32-0000000000000000000000/build/testing/mozbase/test.py' failed, return code 1
e:\builds\moz2_slave\try-w32-0000000000000000000000\build\config\rules.mk:1775:0: command 'C:/mozilla-build/python27/python.exe e:/builds/moz2_slave/try-w32-0000000000000000000000/build/build/pymake/pymake/../make.py -C testing/mozbase check' failed, return code 2
make.py[1]: Entering directory 'e:\builds\moz2_slave\try-w32-0000000000000000000000\build\obj-firefox\js/src'
make.py[2]: Entering directory 'e:\builds\moz2_slave\try-w32-0000000000000000000000\build\obj-firefox\js\src\config'
make.py[2]: Leaving directory 'e:\builds\moz2_slave\try-w32-0000000000000000000000\build\obj-firefox\js\src\config'
make.py[2]: Entering directory 'e:\builds\moz2_slave\try-w32-0000000000000000000000\build\obj-firefox\js\src\shell'
"""

So the good news is the suspicions wrt NamedTemporaryFile were correct.

Comment 40

[Jeff Hammel]

I believe at this point (could be wrong) we are just wrongly not determining win32 in the Makefile.  Note that this doesn't happen on win64

Comment 41

[(not currently active) Ted Mielczarek]

Note that the mozcrash tests don't actually run any applications, that PROCESS-CRASH output is just normal test output.

Comment 42

[Jeff Hammel]

Attachment: more changes

hopefully this at least has enough info to figure out what is wrong on windows

https://tbpl.mozilla.org/?tree=Try&rev=de9472be6e83

Comment 43

[Jeff Hammel]

If this yields useful information leading to a quick fix, let's go for it.  Otherwise, let's just disable the mozprocess tests on windows and file a follow-up to fix

Comment 44

[Jeff Hammel]

Beh, and wouldn't you know it, I pushed everything except the one-line change I needed to test

Comment 45

[Jeff Hammel]

(In reply to Jeff Hammel [:jhammel] from comment #44)
> Beh, and wouldn't you know it, I pushed everything except the one-line
> change I needed to test

But actually I did get the information I wanted...it is just surprising:

"""
make[2]: Entering directory `/e/builds/moz2_slave/try-w32-0000000000000000000000/build/testing/mozbase/mozprocess/tests'
compiling proclaunch; platform: MINGW32_NT-6.1
cl //Od //I "iniparser" //D "WIN32" //D "_WIN32" //D "_DEBUG" //D "_CONSOLE" //D "_UNICODE" //D "UNICODE" //Gm //EHsc //RTC1 //MDd //W3 //nologo //c //ZI //TC proclaunch.c
proclaunch.c

proclaunch.c(23) : warning C4996: '_swprintf': This function or variable may be unsafe. Consider using _swprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(539) : see declaration of '_swprintf'

proclaunch.c(96) : warning C4996: 'strtok': This function or variable may be unsafe. Consider using strtok_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\string.h(166) : see declaration of 'strtok'

proclaunch.c(106) : warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(366) : see declaration of 'sprintf'

proclaunch.c(107) : warning C4996: 'sprintf': This function or variable may be unsafe. Consider using sprintf_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\stdio.h(366) : see declaration of 'sprintf'

proclaunch.c(120) : warning C4996: 'strtok': This function or variable may be unsafe. Consider using strtok_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.

        C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\INCLUDE\string.h(166) : see declaration of 'strtok'

proclaunch.c(126) : warning C4101: 'cmd' : unreferenced local variable

link //OUT:"proclaunch.exe" //INCREMENTAL //LIBPATH:"iniparser\\" //NOLOGO //DEBUG //SUBSYSTEM:CONSOLE //DYNAMICBASE //NXCOMPAT //MACHINE:X86 //ERRORREPORT:PROMPT iniparser.lib  kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib proclaunch.obj
proclaunch.obj : fatal error LNK1112: module machine type 'x64' conflicts with target machine type 'X86'

ok
ERROR
"""

Note: compiling proclaunch; platform: MINGW32_NT-6.1 ; compare: https://github.com/mozilla/mozbase/blob/master/mozprocess/tests/Makefile#L5 ;

So this is actually getting set correctly.  Something else is wrong.

I vote to:
- disable mozprocess windows tests for this landing and file a follow up bug
- land all the code I've written to make this possible on mozbase
- do some mozbase -> m-c mirroring
- close this bug at long last

Comment 46

[Jonathan Griffin (:jgriffin)]

This error seems to imply that one of the libs we're linking against is 64-bit.  Despite the fact that we're testing on "MINGW32_NT-6.1", the OS itself might be 64-bit.

IMO the problem is likely that we're creating a 64-bit version of iniparser.lib, which can't be linked to a 32-bit proclaunch.obj.  We might fix this by passing //MACHINE:X86 when linking iniparser.

Comment 47

[Jeff Hammel]

Is it worth trying this now?

Comment 48

[Jonathan Griffin (:jgriffin)]

I think a try job with this change would be a good thing.  If that fails, then land w/o Windows support and file a follow-up bug.

Comment 49

[Jeff Hammel]

Attachment: and even more

I think this is what you mean, Jonathan?

pushed to try:

mozillatry -p win32 -p win64 http://k0s.org/mozilla/hg/mozilla-central-patches/raw-file/tip/bug-872285 http://k0s.org/mozilla/hg/mozilla-central-patches/raw-file/tip/bug-790765

https://tbpl.mozilla.org/?tree=Try&rev=7cdb62516acf

Comment 50

[Jonathan Griffin (:jgriffin)]

Yes, that's what I was thinking, thanks.

Comment 51

[Jonathan Griffin (:jgriffin)]

I just reproduced this problem locally; the builder is indeed a 64-bit machine, using a 64-bit toolchain.  The fix is just to remove the //MACHINE:x86 flag from proclaunch, rather than adding it to iniparser.

I assume this will work for xp as well, but haven't tried.

The above try run will likely fail (that solution failed for me locally).

FYI, for building 32-bit Firefox in this environment, our mozconfig sets up alternate paths for LIB and other things:

if test "$PROCESSOR_ARCHITECTURE" = "AMD64" -o "$PROCESSOR_ARCHITEW6432" = "AMD64"; then
  . $topsrcdir/build/win32/mozconfig.vs2010-win64
else
  . $topsrcdir/build/win32/mozconfig.vs2010
fi

But, I think that's probably overkill for this; just letting the the toolchain decide what arch to use seems like the best solution.

Comment 52

[Jonathan Griffin (:jgriffin)]

Also, I get 3 mozprocess test failures, regardless of which arch ends up being used to make proclaunch.exe.  So we may still need to roll these out without Windows and file a follow-up bug, if the same thing happens on try.

Comment 53

[Jeff Hammel]

Wonderful :(  Can you paste the errors?  FWIW, these really need some cleanup.  My patch does some, but there is a lot left to do, preferably by someone that actually knows windows.

Comment 54

[Jonathan Griffin (:jgriffin)]

My local test failures:

======================================================================
ERROR: tearDownClass (__main__.ProcTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_mozprocess.py", line 98, in tearDownClass
    raise OSError("Error(s) encountered tearing down %s.%s:\n%s" % (cls.__module__, cls.__name__, '\n'.join(errors)))
OSError: Error(s) encountered tearing down __main__.ProcTest:
[Error 5] Access is denied: 'f:\\mozilla\\mozbase\\mozprocess\\tests\\proclaunch.exe'

======================================================================
FAIL: test_process_kill (__main__.ProcTest)
Process is started, we kill it
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_mozprocess.py", line 194, in test_process_kill
    p.didTimeout)
  File "test_mozprocess.py", line 237, in determine_status
    self.assertTrue(returncode == 0, "Detected non-zero return code of: %d" % returncode)
AssertionError: Detected non-zero return code of: 572

======================================================================
FAIL: test_process_waittimeout (__main__.ProcTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_mozprocess.py", line 165, in test_process_waittimeout
    ())
  File "test_mozprocess.py", line 245, in determine_status
    self.assertTrue(detected, "Detected process is not running, process output: %s" % output)
AssertionError: Detected process is not running, process output:

Comment 55

[Jeff Hammel]

It sounds like disabling on windows is the way to go for the time being :(

Comment 56

[Jonathan Griffin (:jgriffin)]

I agree, in the interest of getting these running elsewhere soon.

Comment 57

[Jeff Hammel]

Attachment: hopefully final form

pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=d38925e12dd2

If this looks good, I'll take out the relevent mozbase parts and put them up for review (r?=:ahal), then mirror (bug 872285) and go from there.

Comment 58

[Jeff Hammel]

Such a field of green n'er did i see.

Comment 59

[Jonathan Griffin (:jgriffin)]

so pretty

Comment 60

[Jeff Hammel]

Attachment: rejigger for mozbase

Of course this doesn't apply cleanly

Comment 61

[Jeff Hammel]

Attachment: pieces needed to make `make check` work on m-c

...and a few makefile cleanups that were in my way

Comment 62

[Andrew Halberstadt [:ahal]]

Comment on attachment 753879 [details] [diff] [review]
pieces needed to make `make check` work on m-c

Review of attachment 753879 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good to me. No idea what the makefile stuff is but if it works in try that is good enough for me

Comment 63

[Jeff Hammel]

FWIW, the Makefile stuff is a partial cleanup trying to get the mozprocess test working on windows.  It still doesn't but I think its a step in the right direction

Comment 64

[Jeff Hammel]

(In reply to Andrew Halberstadt [:ahal] from comment #62)
> Comment on attachment 753879 [details] [diff] [review]
> pieces needed to make `make check` work on m-c
> 
> Review of attachment 753879 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> Looks good to me. No idea what the makefile stuff is but if it works in try
> that is good enough for me

pushed to github: https://github.com/mozilla/mozbase/commit/807c391ee8d75caae4d62a2a12023651c786e0d7

Comment 65

[Jeff Hammel]

Attachment: makefile changes

carrying r+ forward; will test on try to make sure all is good

Comment 66

[Jeff Hammel]

pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=7833e2c98424

Comment 67

[Jeff Hammel]

The OS X 10.6 error appears unrelated.  The OS X 0.7 debug error is with the introduced mozdevice tests;  assuming intermittent and retrying.  

Tests DeviceManager timeout, timeout case. ... DeviceManager DEBUG | sent cmd: testroot
DeviceManager DEBUG | recv'ing...
DeviceManager DEBUG | response: /mnt/sdcard
$>
DeviceManager DEBUG | sent cmd: isdir /mnt/sdcard/tests
DeviceManager DEBUG | recv'ing...
DeviceManager DEBUG | response: TRUE
$>
DeviceManager DEBUG | sent cmd: ver
DeviceManager DEBUG | recv'ing...
DeviceManager DEBUG | response: SUTAgentAndroid Version 1.14
$>
DeviceManager INFO  | removing file: /mnt/sdcard/tests/test.txt
DeviceManager DEBUG | sent cmd: isdir /mnt/sdcard/tests
DeviceManager DEBUG | recv'ing...
DeviceManager DEBUG | response: TRUE
$>
DeviceManager DEBUG | sent cmd: cd /mnt/sdcard/tests
DeviceManager DEBUG | recv'ing...
DeviceManager DEBUG | response: 
$>
DeviceManager DEBUG | sent cmd: ls
DeviceManager DEBUG | recv'ing...
DeviceManager DEBUG | response: test.txt
$>
DeviceManager DEBUG | sent cmd: rm /mnt/sdcard/tests/test.txt
DeviceManager DEBUG | recv'ing...
DeviceManager DEBUG | response: 
DeviceManager DEBUG | Automation Error: Error receiving data from socket. cmd={'cmd': 'rm /mnt/sdcard/tests/test.txt'}; err=connection closed

Not sure what to do about this one.  I'd hate to introduce yet another intermittent orange to the tree.

Comment 68

[Jeff Hammel]

going to go ahead and land.  If it proves to be problematic, we can either back out or make a bug for the orange depending.

Comment 69

[Jeff Hammel]

(In reply to Jeff Hammel [:jhammel] from comment #68)
> going to go ahead and land.  If it proves to be problematic, we can either
> back out or make a bug for the orange depending.

inbound is currently closed for bustage.  I'll plan on landing when it reopens, but if others have opinions on what to do in the interim please note here for consideration

Comment 70

[Jeff Hammel]

tree open, pushed to inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/12f538f501b9

Comment 71

[Phil Ringnalda (:philor)]

Yeah, it would be a really particularly extremely bad idea to introduce an intermittent make check failure, since the only way to determine whether a totally incomprehensible failure like a DeviceManager test failing on OS X (a what? on what?) was an actual regression would be to retrigger, and to retrigger make check you retrigger a build, and as a result not only wait an hour or more before the tests even run, you also trigger ten or twenty hours of pointless other mochitests and reftests and crashtests and talos.

https://tbpl.mozilla.org/php/getParsedLog.php?id=23569347&tree=Mozilla-Inbound with the same "Error receiving data from socket" failure, backed out in https://hg.mozilla.org/integration/mozilla-inbound/rev/011c150e419d

Comment 72

[Phil Ringnalda (:philor)]

https://tbpl.mozilla.org/php/getParsedLog.php?id=23572220&tree=Mozilla-Inbound

Comment 73

[Andrew Halberstadt [:ahal]]

Every time we enable a new suite of tests, there are intermittents. In my experience the only way to enable a new test suite in a timely manner is to disable the intermittents, land and fix later. So can we just disable this particular test and get this bug over and done with?

Comment 74

[Andrew Halberstadt [:ahal]]

Though we might want to figure out https://bugzilla.mozilla.org/show_bug.cgi?id=844288#c55 onwards before re-landing this :/

Comment 75

[William Lachance (:wlach)]

(In reply to Jeff Hammel [:jhammel] from comment #67)
> The OS X 10.6 error appears unrelated.  The OS X 0.7 debug error is with the
> introduced mozdevice tests;  assuming intermittent and retrying.  
> ... 
> DeviceManager DEBUG | sent cmd: rm /mnt/sdcard/tests/test.txt
> DeviceManager DEBUG | recv'ing...
> DeviceManager DEBUG | response: 
> DeviceManager DEBUG | Automation Error: Error receiving data from socket.
> cmd={'cmd': 'rm /mnt/sdcard/tests/test.txt'}; err=connection closed
> 
> Not sure what to do about this one.  I'd hate to introduce yet another
> intermittent orange to the tree.

This test looks kind of broken. Investigating...

Comment 76

[Jeff Hammel]

(In reply to Andrew Halberstadt [:ahal] from comment #74)
> Though we might want to figure out
> https://bugzilla.mozilla.org/show_bug.cgi?id=844288#c55 onwards before
> re-landing this :/

In short, it appears that the ever-time-consuming mozprocess tests are leaving behind a proclaunch binary.  1. they shouldn't; 2. if they do....I would suppose this shouldn't fail; 3. but really, they shouldn't!

Comment 77

[William Lachance (:wlach)]

Filed bug 877811 to track the fix to the devicemanager test.

Comment 78

[Jeff Hammel]

The prolaunch issue is at 877823 which i marked as a dep.  However, my current inclination is to disable this test until we actually have time to work on it.  It is a very important test; that said, there is no time to actually work on it and having these tests running in m-c at least gives us a manner of actually running vs environments we care about (i.e. tryserver).  The overall issue with mozprocess tests is at bug 778267

Comment 79

[Jonathan Griffin (:jgriffin)]

I agree; like ahal said, this is pretty typical for standing up a new suite of tests.  Let's disable as needed and get running what's stable enough to run, then fix-up misbehaving tests and re-enable them.

Comment 80

[Jeff Hammel]

Cool; I filed bug 877864 to this end

Comment 81

[Jeff Hammel]

try run with bumped packages: https://tbpl.mozilla.org/?tree=Try&rev=6231beb7543b

removing the proclaunch bug as a dep since we're disabling the mozprocess test in the initial mirroring

Comment 83

[Jeff Hammel]

Attachment: unbitrot

carrying r+ forward

Comment 84

[Jeff Hammel]

pushed to try: https://tbpl.mozilla.org/?tree=Try&rev=420ac731a56b

Comment 85

[Jeff Hammel]

This looks green to me; although, Android 2.2 Armv6 M(1), I haven't had to look at yet.  Since getting this right is important...I'm going to wait until I get a chance to parse the logs.  Although if anyone wants to push in my absence...+1 from me :)

Comment 86

[Jonathan Griffin (:jgriffin)]

The error is:

DeviceManager: pull unsuccessful: could not get all file data

It looks like this is a case of bug 799334.

I think it extremely unlikely that had anything to do with this patch, and a retrigger was green, so I think it's likely safe to land.

Comment 87

[Jeff Hammel]

(In reply to Jonathan Griffin (:jgriffin) from comment #86)
> The error is:
> 
> DeviceManager: pull unsuccessful: could not get all file data
> 
> It looks like this is a case of bug 799334.
> 
> I think it extremely unlikely that had anything to do with this patch, and a
> retrigger was green, so I think it's likely safe to land.

Thanks, :jgriffin.  I'm going to push and hope for the best

Comment 88

[Jeff Hammel]

pushed to inbound: https://hg.mozilla.org/integration/mozilla-inbound/rev/ad548e637ac1

Comment 89

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/mozilla-central/rev/ad548e637ac1